I have a table definition
class Transaction(
val ...
) extends KeyedEntity[Long] {
val id:Long = 0
}
val transaction = table[Transaction]("transactions")
on(transaction) {t =>
declare(
t.id is unique
... (other fields)
)
}
The database table was not generated by Squeryl (I created it manually), but the "ID" column is set to PrimaryKey and AutoIncrement.
Now I'm inserting a row in this table:
val row = new Transaction(...)
val rowResult = transaction.insert(row)
println("Id1="+row.id+"; Id2="+rowResult.id)
The row is correctly inserted into the database and there an ID is assigned (!=0).
But the application prints "ID1=0; ID2=0" on the command line.
Why? And how do I get the assigned ID?
edit:
I did also try the table definition this way
class Transaction(
val id: Long,
...
) extends KeyedEntity[Long]
Didn't make any differences.
When I remove the declaration
t.id is unique
it works. So the problem is solved.
Is this a bug in squeryl?
Related
I have a table, say 'mytable' that use a "rank" column that is unique. After having created some record where rank is successively rec A(rank=0), rec B (rank=1), rec C (rank=2), rec D (rank=3), rec E (rank=4).
I need to insert a new record that will take an existing rank, say 1, and modify the rank value of the following records accordingly.
The result being : rec A(rank=0), new rec (rank=1), rec B (rank=2), rec C (rank=3), rec D (rank=4), rec E (rank=5).
How can I do this ? Can this be solved with mysql only or should I write some important bunch of code in PHP (Yii2) ?
Assuming no rank is skipped you need to shift existing ranks before saving the new record. To do that you can use beforeSave() method of your ActiveRecord like this:
class MyModel extends \yii\db\ActiveRecord
{
public function beforeSave($insert)
{
if (!parent::beforeSave($insert)) {
return false;
}
if ($insert) { //only if we are saving new record
{
$query = self::find()
->where(['rank' => $this->rank]);
if ($query->exists()) { //check if the rank is already present in DB
//we will create the query directly because yii2
// doesn't support order by for update
$command = static::getDb()->createCommand(
"UPDATE " . static::tableName() .
" SET rank = rank + 1 WHERE rank >= :rank ORDER BY rank DESC",
[':rank' => $this->rank]
);
$command->execute();
}
}
return true;
}
// ... other content of your model ...
}
MySQL allows use of ORDER BY in UPDATE query, that will help us deal with fact that doing UPDATE on table is not atomic and the UNIQUE constraint is checked after each row is updated.
It would be more problematic if there are skipped ranks. In that case you will need to shift ranks only until you hit first skipped rank.
Another option might be creating an before insert trigger on the table that would do the rank shifting.
Note:
It might be a good idea to also implement afterDelete method to shift the ranks in oposite direction when some record is removed to avoid skipped ranks.
Resources:
\yii\db\BaseActiveRecord::beforeSave()
\yii\db\ActiveRecord::updateAllCounters() - replaced with direct update
MySQL triggers
MySQL UPDATE syntax
So I'm having trouble understanding why a resultSet from an executed query will not continue to advance when there's clearly more entries to iterate over. I have a table called all_projects, and when I run this query:
SELECT project_title, created_date, isActive FROM all_projects WHERE project_lead='myUser' ORDER BY created_date DESC;
in my PSQL shell, this is the result I get:
The column data types are: String, Timestamp, and Boolean.
I'm attempting to get each row, create an Array[String] from it, to create, ultimately, a list of an array of strings ( List[Array[String]] )
When I iterate over this with resultSet.next(), it is able to retrieve the first two values, but then when I call next after acquiring the first timestamp value, it fails and returns false.
Below is my code - riddled with a lot of println debug statements to see what happens, stack/and print trace will be at the bottom.
def getAll(userName: String, db: Database): List[Array[String]] = {
val tablesQuery = s"SELECT project_title, created_date, isActive FROM all_projects WHERE project_lead=? ORDER BY created_date DESC;"
var returnResult = new ListBuffer[Array[String]]
db.withConnection { conn =>
val ps = conn.prepareStatement(tablesQuery)
ps.setString(1, userName)
val qryResult = ps.executeQuery()
val columnCount = qryResult.getMetaData.getColumnCount
println("RETRIEVED THIS MANY COLUMNS: " + columnCount)
while (qryResult.next()) {
println("Achieved next in while loop >>>>>>>>>>>")
val row = new Array[String](columnCount)
for (i <- 0 to columnCount - 1) {
println(s"Inserting into Array($i) from the column index(${i + 1})")
if (i < 2) {
println("Tried to get string: ");
row(i) = qryResult.getString(i + 1)
} else { // note I also just tried to keep it all as .getString()
println("Tried to get boolean: ");
row(i) = qryResult.getBoolean(i+1).toString
} // retrieve by column index SQL columns start at 1
println("Row before we move on: " + row.mkString(", "))
if (i <= columnCount - 2) {
println("Called next? -> " + qryResult.next())
}
}
returnResult += row
}
}
returnResult.toList
}
And here is the resulting print stack, which should have been fine, but as you can see, returns false when attempting next() when the cursor is on the first timestamp value.
THIS MANY COLUMNS: 3
Achieved next in while loop >>>>>>>>>>>
Inserting into Array(0) from the column index(1)
Tried to get string:
Row before we move on: Wild Elephants of Mexico, null, null
Called next? -> true
Inserting into Array(1) from the column index(2)
Tried to get string:
Row before we move on: Wild Elephants of Mexico, 2017-08-05 11:00:44.078232, null
Called next? -> false
Inserting into Array(2) from the column index(3)
Tried to get boolean:
[error] o.j.ResultSetLogger - java.sql.ResultSet.getBoolean:
throws exception: org.postgresql.util.PSQLException: ResultSet not positioned properly, perhaps you need to call next.
org.postgresql.util.PSQLException: ResultSet not positioned properly, perhaps you need to call next.
What is happening here?
You are calling qryResult.next() within what you mean to be the handling of a single row, breaking everything. (Boy are you making your life much too difficult.)
A ResultSet represents a query result as a movable pointer to a single row of the query result. You handle rows one at a time, and then call next() only when you have fully completed handling of that row.
Let's make things much much simpler. (I'm just writing this out in a web page, don't have time to check or compile it, so please excuse my boo-boos.)
def handleRow( qryResult : ResultSet, columnCount : Int ) : Array[String] = {
(1 to columnCount).map( i => qryResult.getString(i) ).toArray
}
def getAll(userName: String, db: Database): List[Array[String]] = {
val tablesQuery = s"SELECT project_title, created_date, isActive FROM all_projects WHERE project_lead=? ORDER BY created_date DESC;"
db.withConnection { conn =>
val ps = conn.prepareStatement(tablesQuery)
ps.setString(1, userName)
val qryResult = ps.executeQuery()
val columnCount = qryResult.getMetaData.getColumnCount
println("RETRIEVED THIS MANY COLUMNS: " + columnCount)
var buffer = new ListBuffer[Array[String]]
while (qryResult.next()) {
buffer += handleRow( qryResult, columnCount )
}
buffer.toList
}
}
I am trying to get an SQLAlchemy ORM class to automatically:
either lookup the foreign key id for a field
OR
for entries where the field isn't yet in foreign key table, add the row to the foreign key table - and use the auto generated id in the original table.
To illustrate:
Class Definition
class EquityDB_Base(object):
#declared_attr
def __tablename__(cls):
return cls.__name__.lower()
__table_args__ = {'mysql_engine': 'InnoDB'}
__mapper_args__= {'always_refresh': True}
id = Column(Integer, primary_key=True)
def fk(tablename, nullable=False):
return Column("%s_id" % tablename, Integer,
ForeignKey("%s.id" % tablename),
nullable=nullable)
class Sector(EquityDB_Base, Base):
name = Column(String(40))
class Industry(EquityDB_Base, Base):
name = Column(String(40))
sector_id = fk('sector')
sector = relationship('Sector', backref='industries')
class Equity(EquityDB_Base, Base):
symbol = Column(String(10), primary_key=True)
name = Column(String(40))
industry_id = fk('industry')
industry = relationship('Industry', backref='industries')
Using the Class to Set Industry and Sector
for i in industry_record[]:
industry = Industry(id=i.id,
name=i.name,
sector=Sector(name=i.sector_name))
session.merge(industry)
Result
Unfortunately, when I run this - the database adds individual rows to the sector table for each duplicate use of 'sector_name' - for instance, if 10 industries use 'Technology' as their sector name, I get 10 unique sector_id for each one of the 10 industries.
What I WANT - is for each time a sector name is presented that is already in the database, for it to auto-resolve to the appropriate sector_id
I am clearly just learning SQLAlchemy, but can't seem to figure out how to enable this behavior.
Any help would be appreciated!
See answer to a similar question create_or_get entry in a table.
Applying the same logic, you would have something like this:
def create_or_get_sector(sector_name):
obj = session.query(Sector).filter(Sector.name == sector_name).first()
if not obj:
obj = Sector(name = sector_name)
session.add(obj)
return obj
and use it like below:
for i in industry_record[:]:
industry = Industry(id=i.id,
name=i.name,
sector=create_or_get_sector(sector_name=i.sector_name))
session.merge(industry)
One thing you should be careful about is which session instance is used there in the create_or_get_sector.
In my database I want to synchronize two tables. I use auth_user(Default table provided by Django) table for registration and there was another table user-profile that contain entities username, email, age etc. During the synchronization how to update Foriegn key?
def get_filename(instance,filename):
return "upload_files/%s_%s" % (str(time()).replace('.','_'),filename)
def create_profile(sender, **kwargs):
if kwargs["created"]:
p = profile(username = kwargs["instance"], email=kwargs["instance"])
p.save()
models.signals.post_save.connect(create_profile, sender=User)
class profile(models.Model):
username = models.CharField(max_length = 30)
email = models.EmailField()
age = models.PositiveIntegerField(default='15')
picture = models.FileField(upload_to='get_filename')
auth_user_id = models.ForeignKey(User)
Here in table profile during synchronization all columns are filled except auth_user_id. and there was an error
Exception Value:
(1048, "Column 'auth_user_id_id' cannot be null")
You have to alter your table and change the column auth_user_id_id datatype attribute that allows null.
Something like this:-
ALTER TABLE mytable MODIFY auth_user_id_id int;
Assuming auth_user_id_id as int datatype.(Columns are nullable by default)
How can i fetch all the table name and row count for the specific table from the specific database ?
Result
Table Name , Row Count , Table Size(MB)
---------------------------------------
table_1 , 10 , 2.45
table_2 , 20 , 4.00
ActiveRecord::Base.connection.tables.each do |table|
h = ActiveRecord::Base.connection.execute("SHOW TABLE STATUS LIKE '#{table}'").fetch_hash
puts "#{h['Name']} has #{h['Rows']} rows with size: #{h['Data_length']}"
end
The question is tagged mysql but you can do it in a DB-agnostic manner via ORM.
class DatabaseReport
def entry_counts
table_model_names.map do |model_name|
entity = model_name.constantize rescue nil
next if entity.nil?
{ entity.to_s => entity.count }
end.compact
end
private
def table_model_names
ActiveRecord::Base.connection.tables.map(&:singularize).map(&:camelize)
end
end
Note that this will skip tables for which you don't have an object mapping such as meta tables like ar_internal_metadata or schema_migrations. It also cannot infer scoped models (but could be extended to do so). E.g. with Delayed::Job I do this:
def table_model_names
ActiveRecord::Base.connection.tables.map(&:singularize).map(&:camelize) + ["Delayed::Job"]
end
I came up with my own version which is also db agnostic.
As it uses the descendants directly it also handles any tables where the table_name is different to the model name.
The rescue nil exists for cases when you have the class that inherits from ActiveRecord but for some reason don't have a table associated with it. It does give data for STI classes and the parent class.
my_models = ActiveRecord::Base.descendants
results = my_models.inject({}) do |result, model|
result[model.name] = model.count rescue nil
result
end
#temp_table = []
ActiveRecord::Base.connection.tables.each do |table|
count = ActiveRecord::Base.connection.execute("SELECT COUNT(*) as count FROM #{table}").fetch_hash['count']
size = ActiveRecord::Base.connection.execute("SHOW TABLE STATUS LIKE '#{table}'").fetch_hash
#temp_table << {:table_name => table,
:records => count.to_i,
:size_of_table => ((BigDecimal(size['Data_length']) + BigDecimal(size['Index_length']))/1024/1024).round(2)
}
end
end