I'm trying to replace the Django-Settings line
'NAME': os.path.join(BASE_DIR , 'db.sqlite3'),
by
'NAME': os.path.join(BASE_DIR , 'db.mysql'),
but it doesn't find it.
The other line with the ENGINE parameter works fine. The problem must be the "os.path.join"-part.
# replace sqlite3 db with mysql
file { '/var/www/mysite/mysite/settings.py':
ensure => present,
}
file_line { 'replace db engine':
path => '/var/www/mysite/mysite/settings.py',
replace => true,
line => "'ENGINE': 'django.db.backends.mysql',",
match => "'ENGINE': 'django.db.backends.sqlite3',",
append_on_no_match => false,
}
file_line { 'replace db name':
path => '/var/www/mysite/mysite/settings.py',
replace => true,
line => "\'NAME\': os.path.join(BASE_DIR , \'db.mysql\'),",
match => "\'NAME\': os.path.join(BASE_DIR , \'db.sqlite3\'),",
append_on_no_match => false,
}
I tried it with \' and without \ .
Can somebody please help?
EDIT:
So if I add something like this beforehand:
class { '::mysql::server':
root_password => 'strongpassword',
override_options => { 'mysqld' => { 'max_connections' => '1024' } }
}
mysql::db { 'mynewDB':
user => 'admin',
password => 'secret',
host => 'master.puppetlabs.vm',
sql => '/tmp/states.sql',
require => File['/tmp/states.sql'],
}
Then I would replace the NAME parameter with "mynewDB"?
Did I understand it correctly?
You need to bear in mind that the match parameter to a file_line resource conveys a regular expression, not a plain string. Puppet uses the Ruby flavor of regular expressions. In that dialect, like in many others, the parentheses (()) are metacharacters, signifying grouping. You must escape them if you want them to be interpreted as literals. Moreover, because Ruby regexes use the same escape character that Puppet strings do, you must also escape the escape character to pass it through Puppet to the underlying regex engine. On the other hand, you do not need to escape single quotes inside a double-quoted string, or vise versa, though doing so should not be harmful.
Example:
file_line { 'replace db name':
path => '/var/www/mysite/mysite/settings.py',
replace => true,
line => "'NAME': os.path.join(BASE_DIR , 'db.mysql'),",
match => "'NAME': os.path.join\\(BASE_DIR , 'db.sqlite3'\\),",
append_on_no_match => false,
}
But that's a bit of a poor design. If you're trying to ensure that the database you want is properly named (regardless of what the actual name should be), then to the greatest degree possible, you should match the line you want to manage in a way that does not depend on the current database name.
I'm not knowledgeable about Django specifics, but if you can rely on only one NAME property being specified in the settings file then you might instead do this:
file_line { 'replace db name':
path => '/var/www/mysite/mysite/settings.py',
replace => true,
line => 'Whatever the line should really be',
match => "\\s*'NAME':.*",
append_on_no_match => false,
}
The match expression there matches a line with an arbitrary amount of leading whitespace, followed by the literal characters 'NAME':, followed by anything.
But you should also consider whether file_line is the right tool for the job at all. It really makes sense only if you need to accommodate some parts of the file being managed outside Puppet, which is an uncomfortable situation, albeit one that sometimes we are stuck with. If possible, though, it is better to allow Puppet to manage the entire file, including its complete contents.
Related
I'm trying to combine two data into one column. I already managed to combine two data in one column. But my problem is I don't know how to do a break line to separate the data. I have tried this way but doesn't work.
This is my code
'NAME_PROG_ENG'.'REMARKS' => ['label' => "Programme Name",'value' => function ($data) {return ($data->NAME_PROG_ENG)."\r\n".nl2br("Previously known as: ".($data->REMARKS));}],
I think you refer to GridView component.
Some explanation:
'NAME_PROG_ENG'.'REMARKS' =>
the array index is the model field name, you can't combine 2 columns just by concatenating the column name. In this way you just define a column in the grid wich is not connected to the model. This is not an error, is just to notice that this does not give you the expected result.
'value' => function ($data) {return ($data->NAME_PROG_ENG)."\r\n".nl2br("Previously known as: ".($data->REMARKS));}
Between $data->NAME_PROG_ENG and $data->REMARKS you put a "\r\n" which is fine, but you apply nl2br() function only to the second part of the string.
Now about Gridview. Text are automatically converted to entities, this is a security feature to avoid script injection from user input. If you plan to use input from an untrusted user, ensure that you clean it using htmlpurifier before saving it in database or before showing it (in your 'value'=>... function)
https://www.yiiframework.com/search?language=en&version=2.0&type=guide&q=htmlpurifier
You can bypass html entity conversion by specifies the output format as raw in grid column option
'format'=>'raw'
So you column definition should be something like
'prog_and_remark_combined' => [
'format' => 'raw',
'label' => "Programme Name",
'value' => function ($data) {
return nl2br(
$data->NAME_PROG_ENG .
"\r\nPreviously known as: " .
$data->REMARKS
);
}
],
Gridview allow a short notation
https://www.yiiframework.com/doc/api/2.0/yii-grid-gridview#$columns-detail
which is like this
<column name>:<fomat>:<label>
your code can be as follow as well
'prog_and_remark_combined:raw:Programme Name' => [
'value' => function ($data) {
return nl2br(
$data->NAME_PROG_ENG .
"\r\nPreviously known as: " .
$data->REMARKS
);
}
],
Last note, if the fields does not contain new line to be converted, just concatenate them without using nltobr() function
return $data->NAME_PROG_ENG . "<br>Previously known as: " . $data->REMARKS
I have these possible urls:
Having country at beginning:
peru/blog/search/my-search
peru/blog/tag/my-tag
peru/blog/my-blog-post
peru/blog/
peru/
Without Country at beginning:
blog/search/my-search
blog/tag/my-tag
blog/my-blog-post
blog/
/
How it works:
As I understand url management there are 2 processes:
When you write an url on the browser. In this case Yii tries to convert this url into a route and params.
When you are creating an url using Yii::$app->urlManager->createAbsoluteUrl, in example.
According to these, I am writing some rules in the urlManager, first the general config:
'urlManager' => [
'class' => 'yii\web\UrlManager',
'enablePrettyUrl' => true,
'showScriptName' => false,
'rules' => []
]
Now the rules:
'<country:peru|france>/<module:[\w\-]+>/tag/<tag:[\w\-]*>' => '<module>/index',
'<country:peru|france>/<module:[\w\-]+>/search/<search:[\w\-]*>' => '<module>/index',
'<country:peru|france>/<module:[\w\-]+>/<slug:[\w\-]*>' => '<module>/index',
'<country:peru|france>/<module:[\w\-]+>' => '<module>/index',
'<module:[\w\-]+>/tag/<tag:[\w\-]*>' => '<module>/index',
'<module:[\w\-]+>/search/<search:[\w\-]*>' => '<module>/index',
'<module:[\w\-]+>/<slug:[\w\-]*>' => '<module>/index',
'<module:[\w\-]+>' => '<module>/index',
As you notice from the rules, I pass a parameter "module" in the url, and I want that one to be used as controller.
In the case of country I had to add some possible matches, if not I was not able to make it work.
With the above rules, It works when I input a "pretty" url on the browser like:
http://example.com/blog/search/my-search
But my issue starts If try to create an url:
Yii::$app->urlManager->createAbsoluteUrl(["blog/index", "module" => "blog"]
Rule: '<module:[\w\-]+>' => '<module>/index'
Url Expected: http://example.com/blog
Url Generated: http://example.com/blog?module=blog
It seems it does not fall in the rule, not sure.
If I try to create an url:
Yii::$app->urlManager->createAbsoluteUrl(['blog/index', 'module' => 'blog', 'slug' => 'my-post'])
Rule: '<module:[\w\-]+>/<slug:[\w\-]*>' => '<module>/index'
Url Expected: http://example.com/blog/my-post
Url Generated: http://example.com/blog/my-post?module=blog
From these 2 cases, I notice it is adding the controller to the url
Questions:
In my rule I use I think it collides with predefined variables like: , . I have tried change it, but still same issue.
In the case of country I had to add possible options: Peru, france to make it work, if not it did not work, how can I make it work without those options?
The url match depends on the amount of query params or does it count controller and action too?
How can I make empty parameters be ignored for the rules, when creating an url?
Why is adding controller to the url?
Is the rules order correct?
I think I found a solution to all my questions:
So far I have not found predefined keywords for rules: <controller>, <action>, <module> are just name of variables, if someone knows something different, please let me know.
Yii URL manager (and probably any URL manager) can only match the amount of parameters we send and match it against rules, not the variable names, so if we send:
http://example.com/peru
http://example.com/es
Yii only understands that we are sending one parameter, so if in the rules we have:
'<language:[\w\-]*>' => 'language/index'
'<country:[\w\-]*>' => 'country/index'
Yii will use the first rule that matches, so in this case would be <language> rule which will match. Then Yii will pass a variable $language with "peru" as value to LanguageControlle, which is wrong.
So in order to help Yii we have to add patterns to help it use correct rule, we could add a pattern to match only any value with 2 characters or specific values list like:
<language:es|en> => 'language/index'
<country:[\w\-]*> => 'country/index'
In this case if we have "peru" as value, it will not match first rule, so it will use second one.
Answered above.
Ignore empty parameters, we can use + instead of * in the rules, in that way empty parameters will not match.
Remove the controller, we need to add a rule at the end:
'' => 'site/index'
Question ordering, it should start with the rules with most parameters and inside that group order them from the less generic to more generic rules.
At the end my rules are:
'<base:es|en>/<module:ideas|user|blog>/tag/<tag:[\w\-]*>' => '<module>/index',
'<base:es|en>/<module:ideas|user|blog>/search/<search:[\w\-]*>' => '<module>/index',
'<base:es|en>/<module:ideas|user|blog>/<slug:[\w\-]*>' => '<module>/index',
'<base:es|en>/<module:ideas|user|blog>' => '<module>/index',
'<base:es|en>/<slug:contact>' => 'site/index',
'<base:es|en>' => 'site/index',
'<module:ideas|user|blog>/tag/<tag:[\w\-]*>' => '<module>/index',
'<module:ideas|user|blog>/search/<search:[\w\-]*>' => '<module>/index',
'<module:ideas|user|blog>/<slug:[\w\-]*>' => '<module>/index',
'<module:ideas|user|blog>' => '<module>/index',
'<slug:contact>' => 'site/index',
'' => 'site/index',
I hope this saves time to someone in the future.
The problem is how you create URL. If <module> is available in route pattern (value in rules array), then you don't need to pass it manually. It will be detected from route.
Sou you should create your URLs like this:
Yii::$app->urlManager->createAbsoluteUrl(['blog/index', 'slug' => 'my-post'])
Yii::$app->urlManager->createAbsoluteUrl(['blog/index']);
I am using Yii2's pretty urls and want to play around with the rules defined in my UrlManager but not finding any documentation as to how I can define variables in the 'pattern' => 'route' rule set. Found some examples like
'urlManager' => [
'enablePrettyUrl' => true,
'showScriptName' => false,
'enableStrictParsing' => false,
'rules' => [
"home" => "site/index",
"login" => "site/login",
"sign-up" => "site/sign-up",
'<controller:[\w-]+>/<id:\d+>' => '<controller>/view',
],
],
But what does the :[\w-]+ or the :\d+ stand for?
What if I wanted, for example, to define a pattern to point to my action which needs two parameters
class MyController extends Controller{
...
public function actionMyAction($param1, $param2){
...
}
}
now I want my web users to type in the url bar www.mysite.com/my-controller/my-action/X-Y where X is the value of $param1 and Y is the value of $param2 and using - as a parameter separator.
Thanks.
[\w-]+ and \d+ are regular expressions, the first indicating any letter or the dash character, repeated one or more times, the section indicating numbers only, repeated one or more times.
In the rule expression, you use <variable name:regex> to put a placeholder for your route that will resolve to variables passed to your controller action.
The rule should look like this if both $param1 and $param2 are numbers.
'my-controller/my-action/<param1:\d+>-<param2:\d+>' => 'my-controller/my-action',
Swap \d for \w if you need letters.
I am completely new to Drupal. I inherited a very ugly and incorrect code, unfortunately. In fact I would like to implement a proper login-with-facebook feature, which was totally mis-implemented. It tried to identify users by their email address, however, for some reason, upon login with Facebook, users logged in with the wrong user. I would like to identify the user based on Facebook ID, however, there was no column for that purpose in the database.
As a result, I have implemented a small script, which added a facebook_id and a facebook_token to the table representing the users. However, these new columns are not seen by the drupal_get_schema function in bootstrap.
If I do this:
$schema = drupal_get_schema("users");
echo var_dump($schema["fields"]);
It shows the fields except the two newly created fields. This way a SchemaCache object is initialized. I assumed that the schema might be cached. So I tried something different:
$schema = drupal_get_schema("users", true);
echo var_dump($schema["fields"]);
to make sure that drupal_get_complete_schema(true) will be called. However, the fields are not seen this way either. Is there a way I can tell Drupal to acknowledge the existence of the two newly created columns? If not: what should I do? Should I remove the two columns from the database table and use db_add_field("users", "facebook_id") and db_add_field("users", "facebook_token") respectively? If so, where should I call these?
Sorry if the question is too simple or I am misunderstanding these technologies, but I have tried to solve this for hours and I am at a loss, because this is my first drupal/bootstrap project and the source-code using these does not help me at all.
EDIT:
Since, at the time of this writing I have not received any answers apart from a tool recommendation which did not address my question, I have continued my research in the area. I removed the columns from the database to create them in a Drupal way. I have implemented this function in user.module:
function user_schema_alter() {
db_add_field('users', 'facebook_id', array(
'type' => 'varchar', //was initially a bigint, but Drupal generated a query which always crashed
'length' => 20,
'not null' => TRUE,
'default' => ".", //was initially -1, but Drupal generated a query which always crashed
));
db_add_field('users', 'facebook_token', array(
'type' => 'varchar',
'length' => 300,
'not null' => TRUE,
'default' => 'unavailable',
));
}
and I invoke it from altconnect.module, like this:
$schema = drupal_get_schema("users");
if (!isset($schema["fields"]["facebook_id"])) {
user_schema_alter();
}
It creates the columns, but later the existence of those columns will not be known about and subsequently an error will be thrown as the code will try to re-create them. Besides the fact that I had lost a lot of time until I realized that Drupal is unable to support bigint fields having -1 as their default value I had to conclude that with this solution I am exactly in the same situation as I were initially, with the difference that with this Drupal solution I will always get an exception if the columns already exist, because the schema will not be aware of them and subsequently, the code will always enter that if.
I fail to understand why is this so difficult in Drupal and I totally fail to understand why trying
db_add_field('users', 'facebook_id', array(
'type' => 'bigint',
'length' => 20,
'not null' => TRUE,
'default' => -1,
));
throws an exception due to syntax error. Maybe I should just leave this project and tell anyone who considers using Drupal to reconsider :)
I was able to find out what the answer is, at least for Drupal 6.
In user.install we need to do the following:
//...
function user_schema() {
//...
$schema['users'] = array(
//...
'fields' => array(
//...
'facebook_id' => array(
'type' => 'varchar',
'length' => 20,
'not null' => TRUE,
'default' => ".",
),
'facebook_token' => array(
'type' => 'varchar',
'length' => 300,
'not null' => TRUE,
'default' => 'unavailable',
),
//...
),
//...
}
//...
/**
* Adds two fields (the number is some kind of version number, should be the biggest so far for the module)
*/
function user_update_7919() {
db_add_field('users', 'facebook_id', array(
'type' => 'varchar',
'length' => 20,
'not null' => TRUE,
'default' => ".",
));
db_add_field('users', 'facebook_token', array(
'type' => 'varchar',
'length' => 300,
'not null' => TRUE,
'default' => 'unavailable',
));
}
When this is done, log in with the admin user and go to http://example.com/update.php
There you will see the thing to be updated. Run it. If you wonder why do we have to do all this, why don't we run some scripts directly, then the answer is that this is how Drupal operates. It simplifies your life by making it complicated, but do not worry, while you wait for update.php to do the updates which would take less than a second if it was your script, you can ponder about the meaning of life, quantum-mechanics or you can try to find out the reason this is so over-complicated in Drupal and you can go out for a walk. When you focus again, if you are lucky, update.php has completed its job and the two columns should be in the database.
Being the sports nerd that I am, I'm looking to take daily XML files that are produced by the Major League Baseball Website, and import them into either an Access or MySQL database. The issue I'm running into, is that almost every XML file they produce is just slightly different than the last. For example, one game file may have a field named batter23 that is next to event22 while the other file calls it batter24and is next to pitcher25. I know that XML files can be inconsistent, but I know there has to be a way to consistently get the data into a database. Is there anyway to standardize these XML files? Some code that will parse each file in a list, and organize them into a specific style and giving them consistent field names? Currently I import the XML file into a Excel sheet first, where I change the file type to a CSV, but from there the field names and column locations are still different from file to file.
My goal is to have all the files in a structure where I can quickly import them into a database each day, without having to manually change column locations, or field names. I'm open to any and all options, but my experience in most languages are rookie level at best, so forgive me for my lack of knowledge.
The files are pretty standard as far as XML goes.., you just have to figure what each file represents.
I did a quick look around a Red Sox v Royals game from September 14. (Go Sox!)
In year_2014/month_09/day_14/gid_2014_09_14_bosmlb_kcamlb_1/players.xml
I can see that Ortiz has an id of 120074.
If I look in batters for his player Id, I can see his stats for that game.
(year_2014/month_09/day_14/gid_2014_09_14_bosmlb_kcamlb_1/batters/120074.xml)
It goes on. Basically, in order to load these files into a database, you will have perform some level of processing for them to make any sense.
The IDs don't appear to change between games, but I only took a cursory glance.
As for loading the data, XML::Simple in perl can take an XML and spit out a perl data structure very easily. Unless you need something more heavy duty, this should cover you.
Loading the players.xml:
#!/bin/env perl
use strict; use warnings;
use Data::Dumper;
use XML::Simple;
my $players_xml = XMLin('players.xml');
print Dumper $xml;
Gives you something like:
$VAR1 = {
'venue' => 'Kauffman Stadium',
'date' => 'September 14, 2014',
'team' => {
'Boston Red Sox' => {
'id' => 'BOS',
'player' => {
'605141' => {
'avg' => '.283',
'team_abbrev' => 'BOS',
'parent_team_id' => '111',
'hr' => '4',
'team_id' => '111',
'status' => 'A',
'last' => 'Betts',
'rl' => 'R',
'parent_team_abbrev' => 'BOS',
'first' => 'Mookie',
'rbi' => '12',
'game_position' => '2B',
'num' => '50',
'position' => '2B',
'current_position' => '2B',
'boxname' => 'Betts',
'bats' => 'R',
'bat_order' => '1'
},
...
It's then trivial to navigate these hashes and insert DB rows as you like.