I have in my database a column with the parameters value of an url. I want with an sql query to put those parameters in different columns. I give an example:
I have now a column named parameters with for example this value: pOrgNum=j11000&pLanguage=nl&source=homepage
now I want three columns: pOrgnum | pLanguage | source with the values of my parameters.
The problem is that I don't know the order of my parameters or the length of it, so I can't use for example substring(parameters,9,6) to extract the parameter pOrgnum. can someone help me please?
There's a MySQL UDF that you can use to do exactly this, which also handles decoding the params and handles most character encodings, etc.
https://github.com/StirlingMarketingGroup/mysql-get-url-param
Examples
select`get_url_param`('https://www.youtube.com/watch?v=KDszSrddGBc','v');
-- "KDszSrddGBc"
select`get_url_param`('watch?v=KDszSrddGBc','v');
-- "KDszSrddGBc"
select`get_url_param`('watch?v=KDszSrddGBc','x');
-- null
select`get_url_param`('https://www.google.com/search?q=cgo+uint32+to+pointer&rlz=1C1CHBF_enUS767US767&oq=cgo+uint32+to+pointer&aqs=chrome..69i57.12106j0j7&sourceid=chrome&ie=UTF-8','q');
-- "cgo uint32 to pointer"
select`get_url_param`('/search?q=Na%C3%AFvet%C3%A9&oq=Na%C3%AFvet%C3%A9','q');
-- "Naïveté"
Disclaimer, I am the author.
I achieved this by taking the right of the string after the search parameter, then the left of the resulting string before the first &.
This handles
if the parameter was the last in the url (so no "&" follows it)
if the parameter does not exist (returns blank)
varying lengths of the search string (provided you replace "utm_medium" everywhere)
This finds the value of "utm_medium" in a parameter named url:
IF(locate("utm_medium", url)=0, '', LEFT(RIGHT(url,length(url)-locate("utm_medium",url)-length("utm_medium")),IF(locate("&",RIGHT(url,length(url)-locate("utm_medium",url)-length("utm_medium")))=0,length(RIGHT(url,length(url)-locate("utm_medium",url)-length("utm_medium")+1)),locate("&",RIGHT(url,length(url)-locate("utm_medium",url)-length("utm_medium"))))-1)) utm_medium
To use, find and replace url with your field name, and utm_medium with your url parameter.
May be inefficient, but gets the job done, and couldn't find an easy answer elsewhere
Its code work in mysql:
SELECT substring_index(URL_FIELD,'\',-1) FROM DemoTable;
Related
Using MySQLAdmin. Moved data from Windows server and trying to replace case in urls but not finding the matches. Need slashes as I don't want to replace text in anything but the urls (in post table). I think the %20 are the problem somwhow?
UPDATE table_name SET field = replace(field, '/user%20name/', '/User%20Name/')
The actual string is more like:
https://www.example.com/forum/uploads/user%20name/GFCI%20Stds%20Rev%202006%20.pdf
In a case you are using MariaDB you have REGEXP_REPLACE() function.
But best approach is to dump the table into the file. Open it in a Notepad ++
and run regex replace like specified on a pic:
Pattern is: (https:[\/\w\s\.]+uploads/)(\w+)\%20(\w+)((\/.*)+)
Replace with: $1\u$2\%20\u$3$4
Then import the table again
Hope this help
If its MariaDB, you can do the following:
UPDATE table_name SET field = REGEXP_REPLACE(field, '\/user%20name\/', '\/User%20Name\/');
First, please check, what is actually stored in the database: %20 is a html-entity which represents a whitespace. Usually, when you are storing this inside the database, it will be represented as an actual whitespace (converted before you store it) -> Hence your replace doesn't match the actual data.
The second option that might be possible - depending on what you want to do: You are seeing the URL containing %20, therefore you created your database records (which you would like to fetch) with that additional %20 - And when you now try to query your results based on the actual url, the %20 is replaced with an "actual" whitespace (before your query) and hence it doesn't match your stored data.
I'm trying to get my first select to work using selectall_hashref from the Perl DBI module. I've opened a connection to the database (MySQL) successfully. I'm getting an error when I execute the following:
$dbh->selectall_hashref('SELECT id FROM users WHERE login=?',undef,"myusername");
DBI::st=HASH(0x1505a60)->_prepare(...): attribute parameter 'myusername' is not a hash ref at /usr/lib/x86_64-linux-gnu/perl5/5.20/DBD/mysql.pm line 238.
My table should be able to support this query, it has an id column and login column for each user.
The examples I've found for selectall_hashref show the ? substitution parameter being passed as the third parameter. The DBI documentation says that the second and third arguments should be %attr and #bind_values but doesn't give much documentation about them or show working examples.
What is causing the error, and more importantly how do you actually use the %attr and #bind_values correctly?
If you want to store everything as an arrayref where each row is a hashref (which is what your comment seems to indicate), you can use the selectall_arrayref() method with the Slice attribute:
$dbh->selectall_arrayref('SELECT id FROM users WHERE login=?', {Slice => {}}, 'myusername');
It's a little weird, but here's how it works:
If $slice is a hash reference, fetchall_arrayref fetches each row as
a hash reference. If the $slice hash is empty then the keys in the
hashes have whatever name lettercase is returned by default. (See
"FetchHashKeyName" attribute.) If the $slice hash is not empty,
then it is used as a slice to select individual columns by name. The
values of the hash should be set to 1. The key names of the returned
hashes match the letter case of the names in the parameter hash,
regardless of the "FetchHashKeyName" attribute.
It's a good idea to set the FetchHashKeyName attribute on the database handle to make your hash key names consistent; I happen to like NAME_lc in my applications.
The methods expects key column as the second parameter and attributes ref is passed as third one. In the result it builds a hash with the specified column as a key. What you probably want, is selectall_arrayref:
$ dbh->selectall_arrayref('SELECT id FROM users WHERE login=?',undef,"myusername");
I`m working on a data extraction from MySQL server and in several situations I'm using onquery substitutions. In example if I have a null field, it will assign an empty string for some fields, like:
ifnull(`negotiation_type`, "") AS negotiation_type
Now my need is to make similar test, but using substrings. I have a url field, and based on it`s value I want to set another field value, called property_type.
All the URLs will have the substring house or apartment, like:
http://www.example.com/?pics=true/tree-house-over-the-tree
http://www.example.com/?pics=true/blue-apartment-under-the-tere
And I`m wondering if composing the LOCATE function with any other mySQL function I will be able to make the property_type value assignment on SELECT.
Anyone have an idea?
Thanks
This one would do it I presume:
SELECT IF(LOCATE('HOUSE', `url`) > 0, 'HOUSE',IF(
LOCATE('APPARTEMENT', `url`) > 0, 'APPARTEMENT', 'OTHER')) AS property_type
Situation:
I have user model. attribute "meta_data" in db represents "text" type field.
In model it seriazized by custom class. ( serialize :meta_data, CustomJsonSerializer.new )
It means, when I have an instance of user, I can work with meta_data like with Hash.
User.first.meta_data['username']
Problem:
I need to write a search function, which will search users by given string. I can do it by manual building search query in rails ex. User.where("email LIKE '%#{string}%'")...
But what about meta_data ? Should I search in this field by LIKE statement too? If I will do so, it will decrease relevance of found record.
For example:
I have 2 users. One of them has username "patrick", another one is "sergio"
meta data in db will look like this:
1) {username: patrick}
2) {username: sergio}
I want to find sergio , I enter a search string "ser" => but I have 2 results, instead of one. This meta_data string "{uSERname: Patrick}" also has "ser", so it makes this record irrelevant.
Do you have any idea how to solve it?
That's really the problem with serialized data. In theory, the serialization could be an algorithm that is very unsearchable. It could do a Hoffman encoding, or other compression, and store the serialization in binary. You are relying on the assumption that the serialization uses JSON and your string will still be findable as a sub-string in the serialization.
Then the problem you are having is another issue. Other data in the serialization can mess up your results.
In general, if you serialize data, you are making a choice to not be searchable.
So a solution would be to add an additional field that you populate in a way that you control. Have a values field and store a pipe (|) delimited value that you can search. So if the data is {firstname: "Patrick", lastname: "Stern"}, your meta_values field might be "Patrick|Stern".
Also, don't use the where method with a string with #{} expansion of input values. The makes it vulnerable to SQL attacks. Instead use:
where("meta_values is like :pattern", pattern: "%#{string}%")
I know that may not look very different, but ActiveRecord will go through a sanitizing this way. If someone has a semi-colon in string, then ActiveRecord will escape the semi-colon in the search condition.
I have been working with Symfony2 and doctrine2 recently and have realized a peculiar datatype called DC2Type:array that certain Symfony2 Roles get saved as. To me it just looks like a serialized PHP array where a signifies the total number of elements, i is the array index.
The value looks like this:
a:15:{i:0;s:32:"ROLE_SONATA_USER_ADMIN_USER_EDIT";i:1;s:32:"ROLE_SONATA_USER_ADMIN_USER_LIST";i:2;s:34:"ROLE_SONATA_USER_ADMIN_USER_CREATE";i:3;s:32:"ROLE_SONATA_USER_ADMIN_USER_VIEW";i:4;s:34:"ROLE_SONATA_USER_ADMIN_USER_DELETE";i:5;s:36:"ROLE_SONATA_USER_ADMIN_USER_OPERATOR";i:6;s:34:"ROLE_SONATA_USER_ADMIN_USER_MASTER";i:7;s:33:"ROLE_SONATA_USER_ADMIN_GROUP_EDIT";i:8;s:33:"ROLE_SONATA_USER_ADMIN_GROUP_LIST";i:9;s:35:"ROLE_SONATA_USER_ADMIN_GROUP_CREATE";i:10;s:33:"ROLE_SONATA_USER_ADMIN_GROUP_VIEW";i:11;s:35:"ROLE_SONATA_USER_ADMIN_GROUP_DELETE";i:12;s:37:"ROLE_SONATA_USER_ADMIN_GROUP_OPERATOR";i:13;s:35:"ROLE_SONATA_USER_ADMIN_GROUP_MASTER";i:14;s:10:"ROLE_ADMIN";}
I want to know what this datatype is?
And what do the following identifier signifies:
s:
I have searched the internet but haven't got any useful data.
I also bumped upon this cookbook entry - http://readthedocs.org/docs/doctrine-orm/en/2.0.x/cookbook/mysql-enums.html but didn't figure out the origin.
This is not a data type. You might have noticed that the column type is LONGTEXT. DC2Type:array is a comment of the field.
Doctrine uses the field's comment as column's metadata storage place. Since Mysql does not allow you to store an array, Doctrine use DC2Type:array as comment in order to know how to unserialize the content.
Take a look at the link below.
https://github.com/doctrine/dbal/issues/1614
From the link you mentioned, you can see that the comment DC2Type:enumvisibility indicates that the content of the field is a flag, indicating that the record is visible or not. It is not a new data type at all. It should be considered an helper strategy in the database level. For Doctrine, it's a custom data type.
This is simply a string. Its format is a serialized PHP array. The s: refers to the size or length of each item value in the array.
e.g. s:32:"ROLE_SONATA_USER_ADMIN_USER_EDIT"
If you count the characters in the ROLE string, there are 32.
Hope this helps.