I have a stringified JSON column in BigQuery that looks like this:
{"status":0,"buyEquipment":false,"created":{"_seconds":1603578346,"_nanoseconds":909000000},"total":0,"events":{"1603578346909":{"status":0}},"approvalStatus":0,"userId":"xAdkXoah6LMXI4xy9SZ14NUGJIH3","facilityId":"-MF_DJYYyRKbW4fs91v_","orderSize":0,"facility":{"name":"Data Center ehf.","photo":"-MF_cjRQ3dWW3NRMJC6I","province":"Southern Peninsula","city":"Reybaer"},"priceKWh":0.01}
I am attempting to extract certain values from it like so:
SELECT
JSON_EXTRACT(data, '$.created') AS Date_Created
FROM table
And I get something like this:
{"_seconds":1607095273,"_nanoseconds":847000000}
Because it is clearly nested. I am not familiar with how the database being imported from (Firestore) handles timestamp objects, but this is how it is being imported into BigQuery.
I would like to either
Turn this into a more familiar Timestamp format using a single BigQuery or Standard SQL command if possible,
or if not just extract it to a format where it is an easy conversion to human readable date in BigQuery.
I have tried various BigQuery builtin functions to no avail. Thank you!
I would like to ... just extract it to a format where it is an easy conversion to human readable date in BigQuery.
Use below (quite self-explanatory)
select
date(timestamp_seconds(cast(json_extract(data, '$.created._seconds') as int64))) AS Date_Created
from table
if applied to sample data in your question - output is
Related
I have a json data like this
{"0":"6","1":"5","2":"10"}
And on the DB I have table which contains json datas like these
{"0":"6","1":"4"}
{"0":"5","1":"2","2":"7"}
{"0":"3","1":"10","2":"4"}
{"0":"6","1":"5","2":"10","3":"8"}
So, I would like know is it possible or does it make sense to select data by comparing the json datas?
I would like to get any json that may contain any key:value in my input json.
So, from my example they will be these
{"0":"6","1":"4"}
{"0":"6","1":"5","2":"10","3":"8"}
You can use JSON search functions. For example -
SELECT json_field FROM table1
WHERE
JSON_CONTAINS(json_field, '{"0":"6"}')
AND JSON_CONTAINS(json_field, '{"1":"5"}')
AND JSON_CONTAINS(json_field, '{"2":"10"}');
I need to import a large file of csv data into MySQL, and when I attempted to use MySQL's unix_timestamp function to import the dates, about half of the records didn't make it.
As far as I can tell, the datetime values are formatted with either a single first "month" digit or two of them, and the same goes with the day of the month (e.g. 6/6/2014 3:48PM vs. 12/16/2014 3:48PM) This throws off the import completely (well about half of the records won't import).
I'm trying to convert this into a unix_timestamp.
Now I know I could write a script with a regex to fix something like this, but I am wondering is there a simpler way to do a mass import like this? For the record, I am using my text editor to write the sql statements from the csv as "insert into" statements. This is where I tried to use date formatting but it seems to only accept one format.
Any way to do this with such a minor difference in input?
Actually, despite my comment, something like this might work:
COALESCE(STR_TO_DATE(val, "formatcandidate1")
, STR_TO_DATE(val, "formatcandidate2")
, STR_TO_DATE(val, "formatcandidate3")
, STR_TO_DATE(val, "formatcandidate4")
, [etc...]
) AS dateVal
There are online tools to do this kind of stuff
reports.zoho.com is one of them
In this tool you can import data applying a specific date format and skip the other rows.
and you can do the same for all the type of formats that are present in your file
and finally you can export the data with same date format for all the data
ask me any doubts if you have any regarding this :)
I can't understand what format use ArangoDB for date storage.
Attempt to insert date in such format:
{"name": "vasia", "date": date("2013-01-15")}
std.json.JSONException#C:\vibe-d-0.7.24\source\vibe\data\json.d(1116): (1): Error: Expected valid JSON token, got 'date("2013-0'.
It's look like vibed JSON module fail on this string, but what format use Arango?
String in format {"name":"vasia","date":"2013-01-15"} inserting in DB successfully, but I can't understand is it's inserting as text or as Date object?
Is it inserting as text or as
Date object?
As text, because ArangoDB only supports JSON data types. JSON doesn't have a Date type, so dates are usually encoded as strings. How you actually do that is up to you, but since you're using D, I suggest you use Date.toISOExtString. For a few other options, see this question.
I haven't used ArangoDB, but the ArangoDB date documentation suggest you use something like DATE_TIMESTAMP("2013-01-15T14:19:09.522") and / or DATE_ISO8601("2013-01-15T14:19:09.522Z"). Hope this helps.
I have data from my db that i convert to json as the following
[{"name":"partner1","cost":"311328","date":"2013-04-01"},
{"name":"partner1","cost":"363780","date":"2013-04-02"},
{"name":"partner1","cost":"364062","date":"2013-04-03"},
{"name":"partner1","cost":"283128","date":"2013-04-04"},
{"name":"partner1","cost":"322608","date":"2013-04-05"},
{"name":"partner2","cost":"425538","date":"2013-04-01"},
{"name":"partner2","cost":"263670","date":"2013-04-02"},
{"name":"partner2","cost":"213192","date":"2013-04-03"},
{"name":"partner2","cost":"378726","date":"2013-04-04"},
{"name":"partner2","cost":"532698","date":"2013-04-05"}]
How will be the best wat to convert this json data into highcharta standard for multiple series data?
My last resort is to do multiple query to db to construct the data, but i think it will comnsume server time resource.
Appreciate for the help.
regards,
Ideally you'd transform the data to look like this:
{"partner1": [
[Date.UTC(2013,3,1), 311328],
[Date.UTC(2013,3,2), 363780],
...
Note that the month argument to Javascript's Date function starts from 0 (January).
First of all you should use timestamp as dates (i.e by Date.UTC() ), morever values should be number, not string.
I have a csv file that has a date field in a format like (among other fields):
17DEC2009
When I do a mysqlimport, the other fields are imported properly, but this field remains 0000-00-00 00:00:00
How can I import this date properly? Do I have to run a sed/awk command on the file first to put it into a proper format? If so, what would that be like? Does the fact that the month is spelled out instead of a number matter?
STR_TO_DATE() enables you to convert a string to a proper DATE within the query. It expects the date string, and a format string.
Check the examples in the manual entry to figure out the correct format.
I think it should be along the lines of %d%b%Y (However the %b is supposed to produce Strings like Dec instead of DEC so you will have to try out whether it works).
I had this issue in the past. What I had to do was to utilize LOAD DATA and set the appropriate expression here -
[SET col_name = expr,...]
http://dev.mysql.com/doc/refman/5.1/en/load-data.html
Here is the approach I took to solve similar problem. My use case was bit complex with so many columns, but making here simple to present the solution.
I have Persons table with (Id int autogen, name varchar(100),DOB date), and few million of data(name,DOB) needs to be populated from CSV file.
Created additional column in persons table with name like (varchar_DOB varchar(25)).
Imported data using mysqlimport utility into columns(name,varchar_DOB).
Executed update query that updated DOB column using str_to_date(varchar_DOB,'format') function.
Now, I have expected data populated DOB column.
The same logic could be applied in doing even other kind of data formatting like double,time_stamp etc.