i'm trying to fetch all posts and comments from a facebook's page using RestFB. All works, but when i try to fetch a russian page, that has particular chars, storing the result in mysql, every rows has some ? and i understand that encoding isn't good.
So:
My table charset encoding is utf8_general_ci.
From RestFB i fetch feed from page in this way:
Connection<Post> pagePosts = facebookClient.fetchConnection(page+"/feed", Post.class,Parameter.with("message", "utf8"));
but every comments stored in db is always something like:
Liels paldies Amerikas Tirdzniec?bas pal?tai un m?su burv?gajiem viesiem par br?niš??go pas?kumu!
How can i fix?
My problem was in jdbc connection..
Solved in this way:
jdbc:mysql://server/database?characterEncoding=UTF-8&useUnicode=true
Related
and thanks in advance for any help. I'm working on fixing all broken links in a massive WordPress multisite database and need some help writing an SQL query to run via PHP MyAdmin. I've searched, but can't the perfect solution...
PROBLEM: We have more than a thousand broken links that start with http:/ instead of http://
CHALLENGE: The following would result in numerous links starting with http:///
UPDATE wp_1_posts
SET post_content = replace (post_content,
'http:/',
'http://');
PROCESS: I want to write a query to SELECT all these links first, so I can review them to ensure I don't do any damage when replacing the text string. Downloading a db dump and doing a manual S&R is not an option since we're talking about a multi-gigabyte database.
I thought something like this would work...
SELECT * FROM wp_1_posts
WHERE post_content LIKE '%http:/%'
AND WHERE post_content NOT LIKE '%http://%'
But that just throws a syntax error. Am I even close?
QUESTION #1: How can I find all instances of "http:/" without returning all "http://" instances in the query results.
QUESTION #2: How might I safely fix all instances of "http:/" without affecting any "http://" strings.
FYI: I'll admit I know just enough about this to be dangerous, and I am not familiar with regular expressions. at. all. That's why I'm turning to you for help. Thanks again!
This should work, in MYSQL:
UPDATE wp_1_posts SET post_content = replace(post_content,'http:/', 'http://')
WHERE post_content REGEXP 'http:/[^/]'
I am using RODBC to connect mysql with R like below:
library(RODBC)
channel=odbcConnect("MySQL_ODBC_AIRFORECASTSYSTEM",uid="root",pwd = "3896123ray")
sql="select * from region_station"
ttt=sqlQuery(channel,query = sql)`
Ss you can see I've put the result into a data.frame, "ttt", and I can use View(ttt) to show the contain, and the Chinese shows properly.
However, when I use ttt[,2] trying to show the second column in console, it became like this:
Any help would be greatly appreciated.
The first column is the Chinese, the second column is outcome of mysql's hex(), and the third column is the result in Rstudio's console.
二林站 E4BA8CE69E97E7AB99 鈭\x9e\xab\x99
南投站 E58D97E68A95E7AB99 \xe5\x8d\x8a\xab\x99
埔里站 E59F94E9878CE7AB99 \xe5\x9f\x87\xab\x99
大里站 E5A4A7E9878CE7AB99 憭折\x87\xab\x99
彰化站 E5BDB0E58C96E7AB99 敶啣\x8c\xab\x99
忠明站 E5BFA0E6988EE7AB99 敹\x98\xab\x99
950 seems to be Big5. For example
CONVERT(BINARY('大里站') USING big5) --> 憭折
which agrees with one of your dumps.
So...
SET NAMES big5;
(or however you specify the CHARACTER SET to MySQL from Rstudio)
or change the LC values to be utf8.
I've create a form that contains Hindi (UTF-8) data which i want to store in MySQL table. The columns corresponding to UTF data has collation value set to utf_general_ci.
I've successfully stored the data in table but when I'm executing a select-where query, it doesn't returns the data. Here is my query:
SELECT Birth.sno, Birth.bookingnumber, Birth.birth_date, Birth.baby_gender, Birth.baby_name, Birth.baby_father_name, Birth.baby_father_address, Birth.baby_mother_name, Birth.birth_place, Birth.place_type, Birth.applicant_name, Birth.applicant_address, Birth.registration_number, Birth.registration_date, Birth.registration_ward, Birth.registration_city_village, Birth.registration_district, Birth.remark, Birth.mother_place_name, Birth.mother_place_type, Birth.mother_place_district, Birth.mother_place_state, Birth.person_religion, Birth.father_education, Birth.mother_education, Birth.father_occupation, Birth.mother_occupation, Birth.mother_age_at_marriage, Birth.mother_age_at_birth, Birth.count_of_mother_child, Birth.birth_by, Birth.birth_method, Birth.mother_weight_at_birth, Birth.pregnancy_duration, Birth.date_of_issue FROM np.births AS Birth WHERE Birth.baby_name = 'd' AND Birth.baby_father_name = 'e' AND Birth.baby_mother_name = 'f' AND Birth.baby_father_address = 'g' AND Birth.person_religion = 'हिंदू' AND Birth.baby_gender = 'पुरुष'
The name of the database is np and name of the table is births
The above query was printed in the log file. I tried to copy and paste the same query in HeidiSQL (front end for MySQL) but its not running. However, if I remove the following part: ** AND Birth.person_religion = 'हिंदू' AND Birth.baby_gender = 'पुरुष'**, the query works fine.
How can I resolve this issue?
This looks like a case when your MySQL client and your MySQL server do not "talk" the same encoding.
There are 3 places where you need to take care of your encoding.
The Web Form (what the users sees) -> Your Web Application (CakePHP) -> Your Database Server (MySQL)
One of those three is NOT using the same encoding as the others. So by the time:
"'हिंदू'" and "'पुरुष'" get to your database they will be something totally different that will not be found in the database.
So, make sure that in your default.ctp file you have set your encoding:
echo $this->Html->charset(); //this will result in a UTF-8 encoding of the page.
Look at the source code of your web page (where I guess you have a search/filter form).
At the top you should see:
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
Then look for the code generated for your search/filter form. You should see:
<form id="your_form_id" accept-charset="utf-8" method="post" action="/your/action/">
The important part is that it that "utf-8" that MUST show up in those places.
Next, look into your database.php file and make sure this line:
'encoding' => 'utf8', is NOT commented out!
Finally, with a client that you are sure supports UTF-8 (probably HeidiSQL) have a look at your data table np.births and make sure that what data you have there actually makes sense! It's possible it got mangled because of the discrepancies in encoding before.
Once the data makes sense in the database you should be good to go!
IF this does not do it you, you'll have to read and thoroughly understand this article. Only then you will be able to locate where the problem is and get your encodings in sync.
(Obviously your PHP source files should be UTF-8 encoded as well...)
I'm in over my head as always, but it's the only way I learn. Right now I am trying to query a column in a database for the current user and return the values. I'm using something like:
#tags = current_user.tags.select(:name).each { |p| p.name}
But it returns:
%5B%23%3CTag+name%3A+%22tag1%22%3E%2C+%23%3CTag+name%3A+%22tag2%22%3E%2C+%23%3CTag+name%3A+%22tag+test%22%3E%5D
From what I understand is that's Url Encoding. Is it possible to clean that up? I've tried using .delete or .gsub but I must be doing something wrong. Any insight? All my research on the site yields how to URL encode, but not URL decode.
For URI encoding/decoding you can take a look at rubyonrails.org:URI::Escape
For displaying HTML in Rails views check out the raw() method rubyonrails.org:ActionView::Helpers::OutputSafetyHelper
I'm currently in hour two of this issue, I can't explain it so I will simply show what is going on. I don't know if this matters at all, but I am using the linkedIN API to retrieve a user's linkedIn unique ID.
In English, what I'm doing:
User Signs in with LinkedIn
I read-in user's LinkedIn ID (returned from the API)
If ID exists in database, say "hello", if not, show them a form to register
The issue I am having:
The following line works and properly returns the 1 user I have in the database with a linkedIn ID of OtOgMaJ2NM
$company_data = "SELECT * FROM s_user WHERE `LI_id` = 'OtOgMaJ2NM'";
The following query returns no results - using the same database with the same record in the table s_user:
$linkedIn_id = "<?js= id ?>";
echo $linkedIn_id;
The following code outputs OtOgMaJ2NM with no trailing spaces.
So far so good ... expcept when I run the query this time using the variable, no records are returned!
$company_data = "SELECT * FROM s_user WHERE `LI_id` = '$linkedIn_id'";
Further notes:
When I echo $company_data the same query is displayed when I use the variable as did when I used the plain text version of the query.
Anyone have ANY ideas?
Thanks,
Evan
I can only assume that when echoing variables it strips the tags, so when you're using it with the query you're actually saying:
$company_data = "SELECT * FROM s_user WHERE `LI_id` = '<?js= OtOgMaJ2NM ?>'";
I could be wrong, but have you tried stripping the tags from the variable?
If you send the variable between the "", the MySQL engine will search for $linkedIn_id literally and not for its content.
Seems you are using php, but I'm not sure about the right syntax. Take a look in the docs.