Seeding NULL values into NOT NULL columns with CI4 - mysql

I'm having a problem with the CodeIgniter 4 seeder that I can't find any solution to so far. I used CodeIgniter 4 migrations to generate my tables and by default the columns are NOT NULL. There were no issues seeding the db in MySQL, even though I only inserted in some columns, which would have made anything else NULL.
The problem is that if I change to PostgreSQL, the migrations run fine. I just needed to change DOUBLE to NUMERIC. The seeder was not so easy. The first error I get is Message: pg_query(): Query failed: ERROR: null value in column "created_at" violates not-null constraint which is weird, since when using MySQL those columns are filled normally. But even if I manually tell my migrations to fill create_at and updated_at, I keep getting this error in any other columns that are NOT NULL, while in MySQL they were simply left empty.
I think I'm misunderstanding something with CI4, MySQL or PostgreSQL in this situation, and would be glad if someone could shed some light.
So to sum this up, I tried seeding a DB using PostgreSQL, which worked fine in MySQL, and it wouldn't let me not insert any value in the dozens of other fields not specified in the seeder. And I'm worried this will be a problem in forms and this sort of thing, since some fields not being filled will result in the same error.
For some clarification, here is the migration for a table in my DB:
class Configuracoes extends Migration
{
public function up()
{
$this->forge->addField([
'id_config' => [
'type' => 'INT',
'constraint' => 9,
'usigned' => true,
'auto_increment' => true,
],
'nome_do_app' => [
'type' => 'VARCHAR',
'constraint' => 128
],
'tema' => [
'type' => 'INT'
],
'xNome' => [
'type' => 'VARCHAR',
'constraint' => 128
],
'xFant' => [
'type' => 'VARCHAR',
'constraint' => 128
],
'CNPJ' => [
'type' => 'VARCHAR',
'constraint' => 14
],
'telefone' => [
'type' => 'VARCHAR',
'constraint' => 11
],
'endereco' => [
'type' => 'VARCHAR',
'constraint' => 256
],
'arquivo-imagem-de-fundo-login' => [
'type' => 'VARCHAR',
'constraint' => 128
],
'logomarca' => [
'type' => 'VARCHAR',
'constraint' => 128
],
'created_at' => [
'type' => 'DATETIME'
],
'updated_at' => [
'type' => 'DATETIME'
],
'deleted_at' => [
'type' => 'DATETIME'
]
]);
$this->forge->addKey('id_config', true);
$this->forge->createTable('configuracoes');
}
And the Seeder for this table:
$this->db->table('configuracoes')->insert([
'nome_do_app' => 'App',
'tema' => '4',
'xNome' => 'name',
'xFant' => 'name',
'CNPJ' => '0000000000000',
'telefone' => '(000) 0000-0000',
'endereco' => 'adress'
]);
You can see the seeder is only filling some columns, so running it I will get the same error as before, asking for each column missing. I can aways fill the missing columns with blank spaces and it will run just fine, but I need to understand why it's happening and if it will interfere with the POST methods and sorts in my code too.

I learned a thing or two trying to figure this problem out, so migh as well share with anyone interested. Feel free to correct me if I say anything wrong, I will gladly edit my answer.
MySQL and PostgreSQL have a few differences dealing with the NOT NULL constraint, and my code was not considering those differences.
While trying to seed a MySQL database with let's say, this code:
$this->db->table('configuracoes')->insert([
'nome_do_app' => 'App',
'tema' => '4',
'xNome' => 'name',
'xFant' => 'name',
'CNPJ' => '0000000000000',
'telefone' => '(000) 0000-0000',
'endereco' => 'adress'
]);
Anything else left out the seeder will be given a NULL value, and MySQL won't complain about it, even if the fields are defined as NOT NULL. While I'm not sure how it works in depth, this is what I could see during the time trying to resolve this issue.
Another thing that's different about PostgreSQL compared with MySQL, is the date field type. While leaving this field NULL in a MySQL database, it would automatically set it to 0000-00-00 00:00:00:000. But PostgreSQL won't accept such value, neither NULL, of course. Unless you tell Postgre to accept a NULL value in that field, that would be just an empty field like the others. My code wasn't taking this change in consideration either.
With PostgreSQL, if a field is set to NOT NULL, you can't simply tell Postgre to insert a NULL value, not even manually do so. And my entire database was set to NOT NULL, because it would never be a problem to MySQL.
The solution: I reviewed all my migrations, and only left NOT NULL in the fields that actually needed a value no matter what. Then while seeding the table, I took the NOT NULL fields in consideration and manually inserted a default value, like a space or a zero.
Thank you, everyone that tried to help!
PSA: I'm not the one who created this database from the start, I'm just mainteining and continuing a project made by someone else. So I can't really say anything about the choices made to structure it that way.

Related

TYPO3 9 LTS type casting error on mysql time field for TCA type input / dbType time saving empty field

I'm trying to have a time input in TYPO3 9 LTS working together with MySQL 5.7.24.
In the ext_tables.sql the field gets defined like this:
some_field time default NULL
In the TCA the field gets defined like this:
'some_field' => [
'exclude' => 1,
'label' => 'Some field',
'config' => [
'type' => 'input',
'dbType' => 'time',
'eval' => 'time',
],
],
When saving the record in the backend without a time input (which should be possible) I'm getting the error:
These fields of record 1 in table "some_table" have not been saved correctly: some_field! The values might have changed due to type casting of the database.
When looking at the database record the some_field field gets the value 00:00:00 (although the db default is NULL).
When selecting a time the record can be saved and opened without error.
Is this a bug in TYPO3 or how could I fix this behavior?
The bug can be solved by having the following eval:
'eval' => 'time,null',
That means you have given the wrong type for the value on your ext_tables.sql. Additionally, TYPO3 v9 has renderTypes.
Try something like that:
ext_tables.sql
begin int(11) DEFAULT '0' NOT NULL
TCA
'begin' => [
'exclude' => true,
'label' => 'LLL:EXT:your_ext/Resources/Private/Language/locallang_db.xlf:tx_yourext_domain_model_modelname.begin',
'config' => [
'type' => 'input',
'renderType' => 'inputDateTime',
'size' => 10,
'eval' => 'datetime',
'default' => time()
],
],
Additional information!
If you want to display the time in FrontEnd you could use something like that
<f:format.date>{dateObject.begin}</f:format.date>
If you want to modify how it looks, you can use the format attribute as well:
<f:format.date format="%d. %B %Y">{dateObject.begin}</f:format.date>
More about that here: TYPO3 Date format

Newly added field to table is completely ignored

I am completely new to Drupal. I inherited a very ugly and incorrect code, unfortunately. In fact I would like to implement a proper login-with-facebook feature, which was totally mis-implemented. It tried to identify users by their email address, however, for some reason, upon login with Facebook, users logged in with the wrong user. I would like to identify the user based on Facebook ID, however, there was no column for that purpose in the database.
As a result, I have implemented a small script, which added a facebook_id and a facebook_token to the table representing the users. However, these new columns are not seen by the drupal_get_schema function in bootstrap.
If I do this:
$schema = drupal_get_schema("users");
echo var_dump($schema["fields"]);
It shows the fields except the two newly created fields. This way a SchemaCache object is initialized. I assumed that the schema might be cached. So I tried something different:
$schema = drupal_get_schema("users", true);
echo var_dump($schema["fields"]);
to make sure that drupal_get_complete_schema(true) will be called. However, the fields are not seen this way either. Is there a way I can tell Drupal to acknowledge the existence of the two newly created columns? If not: what should I do? Should I remove the two columns from the database table and use db_add_field("users", "facebook_id") and db_add_field("users", "facebook_token") respectively? If so, where should I call these?
Sorry if the question is too simple or I am misunderstanding these technologies, but I have tried to solve this for hours and I am at a loss, because this is my first drupal/bootstrap project and the source-code using these does not help me at all.
EDIT:
Since, at the time of this writing I have not received any answers apart from a tool recommendation which did not address my question, I have continued my research in the area. I removed the columns from the database to create them in a Drupal way. I have implemented this function in user.module:
function user_schema_alter() {
db_add_field('users', 'facebook_id', array(
'type' => 'varchar', //was initially a bigint, but Drupal generated a query which always crashed
'length' => 20,
'not null' => TRUE,
'default' => ".", //was initially -1, but Drupal generated a query which always crashed
));
db_add_field('users', 'facebook_token', array(
'type' => 'varchar',
'length' => 300,
'not null' => TRUE,
'default' => 'unavailable',
));
}
and I invoke it from altconnect.module, like this:
$schema = drupal_get_schema("users");
if (!isset($schema["fields"]["facebook_id"])) {
user_schema_alter();
}
It creates the columns, but later the existence of those columns will not be known about and subsequently an error will be thrown as the code will try to re-create them. Besides the fact that I had lost a lot of time until I realized that Drupal is unable to support bigint fields having -1 as their default value I had to conclude that with this solution I am exactly in the same situation as I were initially, with the difference that with this Drupal solution I will always get an exception if the columns already exist, because the schema will not be aware of them and subsequently, the code will always enter that if.
I fail to understand why is this so difficult in Drupal and I totally fail to understand why trying
db_add_field('users', 'facebook_id', array(
'type' => 'bigint',
'length' => 20,
'not null' => TRUE,
'default' => -1,
));
throws an exception due to syntax error. Maybe I should just leave this project and tell anyone who considers using Drupal to reconsider :)
I was able to find out what the answer is, at least for Drupal 6.
In user.install we need to do the following:
//...
function user_schema() {
//...
$schema['users'] = array(
//...
'fields' => array(
//...
'facebook_id' => array(
'type' => 'varchar',
'length' => 20,
'not null' => TRUE,
'default' => ".",
),
'facebook_token' => array(
'type' => 'varchar',
'length' => 300,
'not null' => TRUE,
'default' => 'unavailable',
),
//...
),
//...
}
//...
/**
* Adds two fields (the number is some kind of version number, should be the biggest so far for the module)
*/
function user_update_7919() {
db_add_field('users', 'facebook_id', array(
'type' => 'varchar',
'length' => 20,
'not null' => TRUE,
'default' => ".",
));
db_add_field('users', 'facebook_token', array(
'type' => 'varchar',
'length' => 300,
'not null' => TRUE,
'default' => 'unavailable',
));
}
When this is done, log in with the admin user and go to http://example.com/update.php
There you will see the thing to be updated. Run it. If you wonder why do we have to do all this, why don't we run some scripts directly, then the answer is that this is how Drupal operates. It simplifies your life by making it complicated, but do not worry, while you wait for update.php to do the updates which would take less than a second if it was your script, you can ponder about the meaning of life, quantum-mechanics or you can try to find out the reason this is so over-complicated in Drupal and you can go out for a walk. When you focus again, if you are lucky, update.php has completed its job and the two columns should be in the database.

Changing selected option in HTML::Form

I am working on an automated form submit script. It is logging in to a vendor's website and populating the fields of a form. When trying to submit, the desired result would be a ticket number displayed, which is acknowledging the form is submitted and the request is processed by their helpdesk.
However the form is not submitted correctly (no acknowledgement is displayed) and I suspect that it is caused by one of the inputs which is a SELECT.
Here is the code I use to set this field:
$forms[3]->value('ProductList','-2');
This has no effect on the the prepared form unfortunately, dumping $forms[3], i see this:
[...]
bless({
'onchange' => ' checkKC(document.all.ProductList, \'~0\'); prodExpand();',
'current' => 1,
'menu' => [
{
'seen' => 1,
'value' => '~0',
'name' => '<Please select>'
},
{
'seen' => 1,
'value' => '-2',
'name' => 'Product not found.... Search more'
},
{
'value' => '-1',
'name' => '------------------------------------'
},
{
'value' => 'Product1',
'name' => 'Product 1 Name'
}
],
'name' => 'ProductList',
'id' => 'ProductList',
'idx' => 1,
'type' => 'option'
}, 'HTML::Form::ListInput' ),
[...]
Am I using the right method of $forms[3]? (it was created by HTML::Form->parse($pageresult) btw) Or is there any other method I should try? I can't find any documentation for HTML::Form::ListInput
Thanks for any advice
Consider using WWW::Mechanize for form processing that takes more than one step. That way you can include the login process in your script along with going to the form and of course getting the result.
Or if you need to work with JavaScript, then use WWW::Mechanize::Firefox.

CakePHP 1.3 not saving to database but sql statement is correct and insertID is increased correctly

I already searched many forums for my really strange issue, but I still can't figure out whats going wrong during my save process... The issue: Cake says, my data was saved, creates an autoincrement-ID but no record is stored in the database.
The environment
I have a cake-1.3.13 app running for some time and now needed to add another database table, which is of course related to other tables. My problem is saving records for the habtm-relation table, which looks like this:
CREATE TABLE IF NOT EXISTS `employees_projects_rejectreasons` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`employees_project_id` int(10) unsigned NOT NULL,
`rejectreason_id` int(10) unsigned NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `employees_project_id` (`employees_project_id`,`rejectreason_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=6;
I scaffolded the simple model only with basic validation criteria.
<?php
class EmployeesProjectsRejectreason extends AppModel {
var $name = 'EmployeesProjectsRejectreason';
var $validate = array(
'employees_project_id' => array(
'numeric' => array(
'rule' => array('numeric'),
//'message' => 'Your custom message here',
//'allowEmpty' => false,
//'required' => false,
//'last' => false, // Stop validation after this rule
//'on' => 'create', // Limit validation to 'create' or 'update' operations
),
),
'rejectreason_id' => array(
'numeric' => array(
'rule' => array('numeric'),
//'message' => 'Your custom message here',
//'allowEmpty' => false,
//'required' => false,
//'last' => false, // Stop validation after this rule
//'on' => 'create', // Limit validation to 'create' or 'update' operations
),
),
);
//The Associations below have been created with all possible keys, those that are not needed can be removed
var $belongsTo = array(
'EmployeesProject' => array(
'className' => 'EmployeesProject',
'foreignKey' => 'employees_project_id',
'conditions' => '',
'fields' => '',
'order' => ''
),
'Rejectreason' => array(
'className' => 'Rejectreason',
'foreignKey' => 'rejectreason_id',
'conditions' => '',
'fields' => '',
'order' => ''
)
);
I created several records for Rejectreasons and EmployeesProjects, so I have some valid entries here in the database. Now I want to link them together by creating a new record in the given employees_projects_rejectreasons table. I try to do this from another controller (the EmployeesProjectsController). Here is my latest attempt to save the data:
$this->EmployeesProject->EmployeesProjectsRejectreason->create();
$eprData = array(
'EmployeesProjectsRejectreason' => array(
'employees_project_id' => (int)$id,
'rejectreason_id' => (int)$rrId
)
);
if($this->EmployeesProject->EmployeesProjectsRejectreason->save($eprData)) {
debug('successfully saved EPR with ID '.$this->EmployeesProject->EmployeesProjectsRejectreason->__insertID);
} else {
debug('could not save EPR with employees_project_id='.$id.' and rejectreason_id='.$rrId);
}
Now what happens
After I make an attempt to save a record, my debug gives me the following success report:
successfully saved EPR with ID 4
So the save() call returned true, a new ID was created by the auto_increment function of mySQL. So far so good. But when I check my database, there was no record created. But the auto_increment_counter was increased by 1, as if a record was stored, but it wasn't.
Running the app with debug-level 2, I can see the generated SQL-statement from cake, which looks perfectly fine to me:
INSERT INTO `employees_projects_rejectreasons` (`employees_project_id`, `rejectreason_id`) VALUES (3, 3)
If I run this statement directly on the sql server, the record ist inserted correctly.
What I already tried
I already tried different approaches with the save procedure. I tried working with setters instead of a data-array:
$this->EmployeesProject->EmployeesProjectsRejectreason->set('employees_project_id', $id);
as well, but it made no difference. After I wrote a custom save-method in the EmployeesProjectsRejectreason-Model, calling it from the controller, but it always produced the same result.
I tried
deleting the model-cache
restarting the server-instances and the server itself
Deleting the table and creating it again
disabling validation in the model
removing the unique foreign-key index
Saving with hard-coded and existing ids as foreign key
Some more strange behaviour
The last tests with hard-coded IDs in my controller code confronted me with more riddles: If I try storing existent foreign_key-IDs, the data is not saved as before. But if both IDs are hardcoded and NOT EXISTING (I used invented IDs 345 AND 567, which are definetely not existing in the database) a record was finally inserted!
Moreover I scaffolded Models, Views and Controllers for the new tables. When I run the scaffolded view "myApp/employees_projects_rejectreasons/add" and add a new record, everything works just fine.
I'm just not able to save the record from other controllers. Since I already have a huge headache, solving this problem, I highly appreciate any hint for a solution!!
Thanks in advance guys!
I finally found a solution to solve the issue. I still don't know, why the save code before did not work, but here is how I changed my code to make it work:
From my form, the data array comes in the following format:
Array
(
[EmployeesProject] => Array
(
[id] => 10
[user_id] => 0
[additional_information] => some comment text
[state] => absage
[Rejectreason] => Array
(
[0] => 1
[1] => 8
)
)
)
I searched for some solutions to save habtm relations in cakePHP directly with one call, but that does not seem to be possible in cake-1.3. So I created this pretty simple save routine in my EmployeesProjectController, which works perfectly fine for me:
if (!empty($this->data)) {
if ($this->EmployeesProject->save($this->data)) {
if(array_key_exists('Rejectreason', $this->data['EmployeesProject'])) {
foreach($this->data['EmployeesProject']['Rejectreason'] as $key => $rrId) {
$this->EmployeesProject->EmployeesProjectsRejectreason->create();
$this->EmployeesProject->EmployeesProjectsRejectreason->set('rejectreason_id', $rrId);
$this->EmployeesProject->EmployeesProjectsRejectreason->set('employees_project_id', $this->data['EmployeesProject']['id']);
if($this->EmployeesProject->EmployeesProjectsRejectreason->save()) {
}
}
}
}
}
Thanks #Yoggi for supporting me solving this issue!

Can I create an initial CodeIgniter db migration from .sql?

I have a CodeIgniter application and I just learned about migrations. It sounds very useful and I would like to start using it. However I already have a rather complex database setup. Can someone suggest a reasonable way to create a reliable initial migration from my MYSQL .sql schema file?
It seems excessive to manually recreate the entire db with dbforge, but perhaps that's what I ought to do.
Bit late reply, but I hacked a quick library that should generate the base migration file for you from your current DB. Its still beta, but works on my systems 40 tables+
https://github.com/liaan/codeigniter_migration_base_generation
L:
Visit https://github.com/fastworkx/ci_migrations_generator
You'll get something like this applications/migration/001_create_base.php.
public function up() {
## Create Table sis_customer
$this->dbforge->add_field(array(
'id' => array(
'type' => 'VARCHAR',
'constraint' => 40,
'null' => FALSE,
),
'ip_address' => array(
'type' => 'VARCHAR',
'constraint' => 45,
'null' => FALSE,
),
'timestamp' => array(
'type' => 'INT',
'unsigned' => TRUE,
'null' => FALSE,
'default' => '0',
),
'data' => array(
'type' => 'BLOB',
'null' => FALSE,
),
'`datetime_reg` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ',
));
$this->dbforge->create_table("sessions", TRUE);
$this->db->query('ALTER TABLE `sessions` ENGINE = InnoDB');
));
public function down() {
### Drop table sessions ##
$this->dbforge->drop_table("sessions", TRUE);
}
I was messing around with SQL files manually and then I discovered Jamie Rumbelow's schema library. It makes schema operation a little bit easier to manage then DBForge but still requires manual entry.
https://github.com/jamierumbelow/codeigniter-schema
Someone came up with a mashup of his base model and schema library and it makes prototyping much faster
https://github.com/williamknauss/schema_base_model_magic
this allows you to automate the CRUD model operations & schema