I want to do a scan on a table on dynamodb using boto, my problem is I want to paginate using the max_results and exclusive_start_key
Actually it looks like the only way to access the LastEvaluatedKey to pass it as exclusive_start_key is to manually keep track of primary keys and pass the last one as exclusive_start_key
But that is not my problem, my problem is I don't know what format (what object type) I should pass to exclusive_start_key it does not accept an int even when the table has an integer hash_key?
According to the documentation, Layer2 implementation of Scan expects either a list or a Tuple as a representation of the Primary Key.
(hash_key,) for a single key table
(hash_key, range_key) for a composed key table
Please note that there also is a (tricky) way to directly read the esk from the Scan generator in Boto.
Related
When using a “Create Object” Action, is it possible to configure an incrementing primary key? That is, each new objects primary key is the most recently created object's primary key +1.
It is possible to configure an incrementing number based on the current highest value found in existing objects. However, I would not recommend using an incrementing integer as your primary key for the object.
Generating an auto-incrementing integer
There are two approaches to generating an auto-incrementing number. Both involve writing a custom Function to create the object within the Action. Within the Function, you can either:
Have an instance of a "Counting" object that contains the value of the highest known id for the object type you are creating. Read that maximum value, add 1, set this max as the value on your new object, and update the value on the base "Counting" object.
Perform an aggregation over your object type to find the highest value, add 1, and set the value on the new object.
This is a useful pattern to leverage in many workflows, such as ticketing applications.
Choosing a Primary Key
As a best practice, primary keys should not be coupled to semantically meaningful information about an object. Your primary key should not contain information that is relevant to user workflows. For example, in a ticketing workflow, ticket number formats may change. Updating the value of primary keys is an unnecessary amount of work, while updating the value of a typical string or integer column is feasible.
Therefore, it's recommended that primary keys are uniquely generated UUIDs stored in string columns (ex: ticket_id), while auto-incrementing numbers are stored in column(s) named after what the number represents (ex: ticket_number).
Generating a UUID
The best practice for generating primary keys when creating new objects via Actions is to create a UUID String.
When using the Create Object rule inside of an Action, you can specify the typeclass actions::generate_uuid on the parameter to generate a random UUID every time the Action is submitted.
To do this inside of a Function-backed Action, you can follow the Functions documentation on adding a dependency. For uuid package, this means adding the following to your package.json:
"dependencies": {
"uuid": "^3.4.0"
},
"devDependencies": {
"#types/uuid": "^3.4.9"
}
If you use newer versions of the uuid package it will fail to execute in Functions because of a feature removed in the library here.
After doing this, make sure to click the small banner in the package.json editor window that allows you to get packages from public npm. By default this is disabled.
Then in your code you can add something like:
import { v4 as uuidv4 } from "uuid";
...
#OntologyEditFunction()
public createObject(): void {
var newObject = Objects.create().myObjectType(uuidv4());
}
When working with JSON in MariaDB it is possible to index single-point values using virtual columns e.g.
ALTER TABLE features ADD feature_street VARCHAR(30) AS (JSON_UNQUOTE(feature->"$.properties.STREET"));
ALTER TABLE features ADD INDEX (feature_street);
Does anybody know whether it is possible to index JSON arrays in the same way so that when querying based on the values of the array members, each array does not have to be scanned?
I can't find anything in the docs which suggests this is possible.
Create a "virtual" column of the element of the JSON column and index it.
https://mariadb.com/kb/en/mariadb/virtual-computed-columns/
The elements of an array inside a JSON string -- That is another matter.
I am new to MSAccess so I'm not sure about this; do I have to have a primary key for every single table in my database? I have one table which looks something like this:
(http://i108.photobucket.com/albums/n32/lurker3345/ACCESSHELP.png?t=1382688844)
In this case, every field/column has a repeating term. I have tried assigning the primary key to every field but it returns with an error saying that there is a repeated field.
How do I go about this?
Strictly speaking, Yes, every row in a relational database should have a Primary Key (a unique identifier). If doing quick-and-dirty work, you may be able to get away without one.
Internal Tracking ID
Some database generate a primary key under-the-covers if you do not assign one explicitly. Every database needs some way to internally track each row.
Natural Key
A natural key is an existing field with meaningful data that happens to identify each row uniquely. For example, if you were tracking people assigned to teams, you might have an "employee_id" column on the "person" table.
Surrogate Key
A surrogate key is an extra column you add to a table, just to assign an arbitrary value as the unique identifier. You might assign a serial number (1, 2, 3, …), or a UUID if your database (such as Postgres) supports that data type. Assigning a serial number or UUID is so common that nearly every database engine provides a built-in facility to help you automatically create such a value and assign to new rows.
My Advice
In my experience, any serious long-term project should always use a surrogate key because every natural key I've ever been tempted to use eventually changes. People change their names (get married, etc.). Employee IDs change when company gets acquired by another.
If, on the other hand, you are doing a quick-and-dirty job, such as analyzing a single batch of data to produce a chart once and never again, and your data happens to have a natural key then use it. Beware: One-time jobs often have a way of becoming recurring jobs.
Further advice… When importing data from a source outside your control, assign your own identifier even if the import contains a candidate key.
Composite Key
Some database engines offer a composite key feature, also called compound key, where two or more columns in the table are combined to create a single value which once combined should prove unique. For example, in a "person" table, "first_name" and "last_name", and "phone_number" fields might be unique when considered together. Unless two people married and sharing the same home phone number while also happening to each be named "Alex" with a shared last name! Because of such collisions as well as the tendency for meaningful data to change and also the overhead of calculating such combined values, it is advisable to stick with simple (single-column) keys unless you have a special situation.
If the data doesn't naturally have a unique field to use as the primary key, add an auto-generated integer column called "Id" or similar.
Read the "how to organize my data" section of this page:
http://www.htmlgoodies.com/primers/database/article.php/3478051
This page shows you how to create one (under "add an autonumber primary key"):
http://office.microsoft.com/en-gb/access-help/create-or-remove-a-primary-key-HA010014099.aspx
In you use a DataAdapter and a Currency Manager, your tables must have a primary key in order to push updates, additions and deletions back to the database. Otherwise, they will not register and you will receive an error.
I lost one week figuring that one out until I added this to the Try-Catch-End Try block: MsgBox(er.ToString) which mentioned "key". From there, I figured it out.
(NB : Having a primary key was not a requisite in VB6)
Not having a primary key usually means your data is poorly structured. However, it looks like you're dealing with summary/aggregate data there, so it's probably doesn't matter.
in my application i want to add to every record in a database table a code which must be unique and randomly generated (so unpredictable for users). I assume that generating this by code (java ee) is a bad idea because it will need to request frequently the database management system (MySQL) to check for the unicity.
Can someone help me to generate this code by SQL like for a variable char (varchar) with a size n.
Thanks for your help.
UUID()
Will this work? Of course you would have to store the value of the UUID as your VARCHAR if that is what you wanted rather than the UUID type.
MySQL can verify the uniqueness of a value if you assign a Unique Index to that field. For instance, if you have a table with the columns name, age, code, and you can set a unique key to code and it will make sure that code is never duplicated.
That way you can generate a unique string/number/whatever in whatever language you want (e.g. Java EE) and then insert that string into the table. MySQL will reject an insertion if the key is already being used.
Here's a link to MySQL documentation on adding keys to the table: http://dev.mysql.com/doc/refman/5.1/en/alter-table.html
Hope that helps.
Good luck.
I'm using before and after insert triggers to generate ids (primary key) of the form "ID_NAME-000001" in several tables. At the moment, the value of the hibernate generator class of these pojos is assigned. A random string is assigned to the object to be persisted and when it's inserted by hibernate, the trigger assigns a correct id value.
The problem with this approach is that I'm unable to retrieve the persisted object because the id only exists in the database, not in the object I just saved.
I guess I need to create a custom generator class that could retrieve the id value assigned by the trigger. I've seen an example of this for oracle (https://forum.hibernate.org/viewtopic.php?f=1&t=973262) but I haven't been able to create something similar for MySQL. Any ideas?
Thanks,
update:
Seems that this is a common and, yet, not solved problem. I ended up creating a new column to serve as a unique key to use a select generator class.
Hope this won't spark a holy war for whether using surrogate key or not. But it's time to open the conversation here.
Another approach would be just, use the generated key as surrogate key and assign a new field for your trigger assigned id. The surrogate key is the primary key. You have the logically named key (such as the "ID_NAME-000001" in your example). So your database rows will have 2 keys, the primary key is surrogate key (could be UUID, GUID, running number).
Usually this approach is preferable, because it can adapt to new changes better.
Say, you have these row using surrogate key instead of using the generated id as natural key.
Surrogate key:
id: "2FE6E772-CDD7-4ACD-9506-04670D57AA7F", logical_id: "ID_NAME-000001", ...
Natural key:
id: "ID_NAME-000001", ...
When later a new requirement need the logical_id to be editable, auditable (was it changed, who changed it when) or transferable, having the logical_id as primary key will put you in trouble. Usually you cannot change your primary key. It's horribly disadvantage when you already have lots of data in your database and you have to migrate the data because of the new requirement.
With surrogate key solution, it'll be easy, you just need to add
id: "2FE6E772-CDD7-4ACD-9506-04670D57AA7F", logical_id: "ID_NAME-000001", valid: "F", ...
id: "0A33BF97-666A-494C-B37D-A3CE86D0A047", logical_id: "ID_NAME-000001", valid: "T", ...
MySQL doesn't support sequence (IMO autoincrement isn't comparable to sequence). It's different from Oracle/PostgreSQL's sequence. I guess that's the cause why it's difficult to port the solution from Oracle database to MySQL. PostgeSQL does.