Related
Suppose we have table person and table phoneand the relation between them is one to many.
I need to retrieve this like result with one query.
[
{
name:"abc",
lname:"def",
phones:[
{
dial_code="+1",
number:"12345667"
},
{
dial_code="+1",
number:"12345667"
}
]
},
{
name:"xyz",
lname:"lmn",
phones[
{
dial_code="+2",
number:"2643525"
}
]
},
{...}
]
I can do this by multiple query like first getting all persons and then get their phones one by one but i think its so weird and need lots of time and reduce performance. and if i get all data by joining table it wouldn't be like this JSON format.
Any idea will be appreciated.
Sorry for my bad English.
First things first, you cannot retrieve the desired result with multiple phone inside each person with one single query.
Now, running the query inside person loop will hugely affect the performance of the script if there are a lot of data. In this way, first, you need to execute a query to fetch all persons(say n persons). Then you have to again loop all n persons to fetch their respective phones.
So you need to run something like following inside $persons loop n times:
SELECT * FROM phone WHERE person_id = [$person_id]
Therefore in this way you need to execute n+1 queries.
To overcome this n+1 query problem we can apply a methodology which is called as eager loading. Here you also need to execute the first query to retrieve all persons and then write a query to fetch all phones which belongs to those retrieved persons:
SELECT * FROM person
Result($persons):
id name
5 John
10 Bob
20 Jenna
SELECT * FROM phone WHERE person_id IN (5,10,20)
Result($phones):
id person_id dial_code number
1 5 +2 12345
2 10 +1 12312
3 20 +1 98765
Now we combine these two results in PHP scripts to produce the desired array. In this way, we write only two queries instead of n+1 queries.
You can write a PHP script like following to combine the two result sets:
// Create an array of phones with person_id as key
$phones_with_person_id_as_key = [];
foreach($phones as $key => $phone) {
$phones_with_person_id_as_key[$phone->person_id][$key] = $phone;
}
// Loop $persons array and add phones to person object
foreach($persons as $key => $person) {
// create phones key and add data
if (!empty($phones_with_person_id_as_key[$person->id])) {
$person->phones = $phones_with_person_id_as_key[$person->id];
}
else {
$person->phones = [];
}
}
Now $persons contains the formatted desired output.
I have a hash which is the result of a .map method on a MySQL2::Result object which looks like this:
{#<Date: 2018-01-02 ((2458121j,0s,0n),+0s,2299161j)>=>"OL,BD,DM,WW,DG"}
{#<Date: 2018-01-03 ((2458122j,0s,0n),+0s,2299161j)>=>"KP,LW"}
{#<Date: 2018-01-04 ((2458123j,0s,0n),+0s,2299161j)>=>"LW,WW,FS,DG"}
{#<Date: 2018-01-05 ((2458124j,0s,0n),+0s,2299161j)>=>"OL,KP,BD,SB,LW,DM,AS,WW,FS,DG"}
{#<Date: 2018-01-06 ((2458125j,0s,0n),+0s,2299161j)>=>"OL,KP,BD,SB,LW,DM,AS,WW,FS,DG"}
I would like to pull the values (the two letter items) from the hash, by referencing with the key.
I have tried
puts hash_name["2018-01-06"]
puts hash_name['2018-01-06']
puts hash_name[Date.new(2018,1,6)]
puts hash_name["<Date: 2018-01-06 ((2458125j,0s,0n),+0s,2299161j)>"]
puts hash_name["#<Date: 2018-01-06 ((2458125j,0s,0n),+0s,2299161j)>"]
All return nothing or an error.
The hash is created by doing the following:
hash_name = #available_items.map do
|h| {h["tdate"] => h["items"] }
end
Is there something I can do during the creation of the hash, or now, to be able to easily pull the value out using e.g. can I convert it to some other date format like ISO format?
Thanks
I think your problem is that Enumerable#map doesn't do what you think it does. This:
hash_name = #available_items.map do
|h| {h["tdate"] => h["items"] }
end
will give you an array of single entry hashes, the individual hashes will map Dates to strings but the result looks like:
[
{ date1 => string1 },
{ date2 => string2 },
...
]
rather than:
{
date1 => string1,
date2 => string2,
...
}
as you're expecting. Switching to #each_with_object should take care of your problem:
hash_name = #available_items.each_with_object({}) do |row, h|
h[row['tdate']] = row['items']
end
You're close here, but you're generating an array of hashes, not a singular hash:
hash_name = #available_items.map do |i|
[ i["tdate"], i["items"] ]
end.to_h
This creates an array of key/value pair arrays, then converts them to a hash with the .to_h method.
You can also use group_by if your input data can be grouped neatly, like:
hash_name = #available_items.group_by do |i|
i['tdate']
end
Where that approach might be good enough if can deal with the output format. It's keyed by date.
Note that using symbol keys like :tdate and :items is usually preferable to string keys. It's worth trying to steer towards that in most cases where there'd otherwise be rampant repetition of those strings.
In the hopes that this may help others to do a similar thing, here is what I ended up doing.
I have a MySQL2::Result object as shown above, on which I run:
#available_hash = #available_items.map do |row|
[ row["tdate"], row["available"] ]
end.to_h
Having previously declared a start_date and an end_date I then select an available item from the list, at random to fill a new hash using the dates as keys:
$final_hash = Hash.new("")
for date in (start_date..end_date)
#available_today = #available_hash[date].to_s.split(",")
$final_hash[date] = random_item(#available_today)
date +=1;
end
Whilst I am sure there is probably a more elegant way of doing this, I am delighted that you have helped me to get this to work!
Obviously hash map is not suitable for a date as the key, hash map is more suitable for key as id, tag, etc. It should be a unique key.
Please provide more information about what you need to do with this hash map, for sure you can have some more clever data structure.
If you have an array with two keys (tdate, items) and you want to lookup for the date just use select:
result = available_items.select { |elem| elem['tdate'] === Date.new(2001,2,3) }
reference for '===' operator in Date class
http://ruby-doc.org/stdlib-2.1.1/libdoc/date/rdoc/Date.html#method-i-3D-3D-3D
I am retrieving data from mysql db. All the data is one column. I need to separate this into several cols: The structure of this col is as follows:
{{product ID=001 |Country=Netherlands |Repository Link=http://googt.com |Other Relevant Information=test }} ==Description== this are the below codes: code 1 code2 ==Case Study== case study 1 txt case study 2 txt ==Benefits== ben 1 ben 2 === Requirements === (empty col) === Architecture === *arch1 *arch2
So I want cols like: Product ID, Country, Repository Link, Architecture etc.....
If you are planning on simply parsing out the output of your column, it will depend on the language of choice you are currently using.
However, in general the procedure for doing this is as follows.
1, pull output into string
2, find a delimiter(In you case it appears '|' will do)
3, you have to options here(again depending on language)
A, Split each segment into an array
1, Run array through looping structure to print out each section OR use array
to manipulate data individually(your choice)
B, In Simple String method, you can either create a new string, or replace all
instances of '|' with '\n'(new line char) so that you can display all data.
I recommend the array conversion as this will allow you to easily interact with the data in a simple manner.
This is often something done today with json and other such formats which are often stored in single fields for various reasons.
Here is an example done in php making use of explode()
$unparsed = "this | is | a | string that is | not: parsed";
$parsed = explode("|", $unparsed);
echo $parsed[2]; // would be a
echo $parsed[4]; // would be not: parsed
I'm trying to import a non-csv data file into MySQL.
1) The data fields are newline delimited, and the field identifier is at the start of each line.
2) Some fields have multiple entries
3) Not every record has every field populated
4) Some blank lines exist inside fields and need to be filtered out
5) Records are generally delimited by blank lines, but also by "Number X"
Here's an example of the file showing an example of three records as they appear
Number 1
ARTIST BOOM JEFF=SINGER
BACKING MUSICIANS=BAND
COMP BOOM JEFF
DATE 1980
TIME 3.23
FIELD3 FRONT ROW
NOTE LIVE RECORDING
Number 2
ARTIST JOHN LEE=VOCAL
COMP JOHN LEE
TIME 4.20
ID 000000230682
PUBLISHER BLAHBLAH
FIELD3 DAY I RODE THE TRAIN
Number 3
ARTIST BURT DAN=NARRATOR
JOHNS RY=DRUMS
STUDIO BAND=ORCHESTRA
FREE DAN=DIRECTOR
COMP JOHNS RY
DATE 1934
DUR 2.32
ID 000055332
PUBLISHER WEEWAH
SHELF 86000002
FIELD3 EVE OF THE WAR
NOTE FROM HE NARRATION "NO MORE THAT IN
THE FIRST YEARS OF THE SEVENTEENTH CENTURY .."
What's the best way to import this data into MySQL?
Can LOAD DATA INFILE be used to read it in? Or should I write a script to strip the field identifiers and convert it to csv format which can then be read in using LOAD DATA INFILE?
I would rather use sed to convert those to INSERT .. SET ... statements like:
INSERT INTO RECORDS SET
ARTIST="BOOM JEFF=SINGER~BACKING MUSICIANS=BAND" ,
COMP="BOOM JEFF" ,
DATE="1980" ,
TIME="3.23" ,
FIELD3="FRONT ROW" ,
NOTE="LIVE RECORDING"
replacing in-record newlines with ~ for example and after that analyse data with the help of SQL.
From what I see, your best bet is a script that would parse the data line by line with a script similar to the following (using php):
$lines=explode("\n",file_get_contents('file.name'));
$record=null;
//go through all the lines
foreach($lines as $line) {
//if the line is not empty, add the field to the record
if(trim($line)) {
//I am only processing the field name-you'll have to do the same for equal signs
$pos = strpos($line, ' ');
$fieldName=substr($line,0,$pos;
$fieldValue=substr($line,$pos+1);
$record[$fieldName]=$fieldValue;
}
//if it is a blank line and we have a record, save it
else if ($record) {
//insert the record into the database
insertRecord($record);
//empty the record as the next line is a new record
$record=null;
}
}
function insertRecord($record) {
//to do implement an sql insert statement
}
I presume a simple question. I have the following data.
I want to search for all rows where the ID is > 2 but < 8 and the Price is > 30
I have used various versions of: startkey=["2", null] or even something like startkey=["2", "30"] just for testing.
It only ever seems to run both conditions on the first row. So if I do: startkey=["2", "30"] then I get back:
{"id":"3","key":["3","30"],"value":null},
{"id":"4","key":["4","30"],"value":null},
{"id":"5","key":["5","20"],"value":null},
{"id":"6","key":["6","60"],"value":null},
{"id":"8","key":["8","60"],"value":null}
Why is row 5 there?
I am starting to get the view that I need to handle this in the code (.net) and make multiple calls somehow... I can't seem to find anything on this that works....
Note: I have tried doing say a loop with for (i = 0; i < doc.ID.length; i++) and then using doc.ID[i] but it never returns anything....
Currently I just have
function (doc, meta) {
emit([doc.ID, doc.Price ],null);
}
Essentially I want to have a search where there are 5 input keys that a user has. So do I need to make 5 calls and then keep taking data from the previous output as the source for the next???
Other references I have looked at include: the manual
Thanks in advance,
Kindest Regards
Robin
This is a common misconception, with a compound array index key, it's still treated as a string, therefore the index key [2,10] is actually "[2,10]", and the index key [5,20], is actually "[5,20]".
So the reason that startkey=["2", "30"]shows the {"id":"5","key":["5","20"],"value":null}, row is because as a string it is > startkey.
Likewise, the Query startkey=[2,10]&endkey=[5,10] returns
{"total_rows":7,"rows":[
{"id":"2","key":[2,20],"value":null},
{"id":"3","key":[3,30],"value":null},
{"id":"4","key":[4,30],"value":null}
]
}
because startkey="[2,10]" < "[2,20]" && "[4,30]" < "[5,10]"=endkey, but "[5,20]" is not within that string Range.
Range Queries with startkey and endkey
startkey => endkey is a Range query using strcmp(), the group and group level is based on the string, where the comma is separating string tokens.
A Good Reference Link (since Couchbase Views work much like Apache CouchDB Views (inspired by them))
http://wiki.apache.org/couchdb/View_collation#Collation_Specification
Spatial View/Query
To achieve the result you are trying for, you could also write a Spatial View to have multi-dimensional Queries, numeric only. While you might not initially think of it
function (doc, meta) {
emit({
type: "Point",
coordinates: [doc.ID, doc.Price]
}, meta.id);
}
The Query would be a Bounding Box Query:
&bbox=2,0,8,30
{"total_rows":0,"rows":[
{"id":"2","bbox":[2,20,2,20],"geometry":{"type":"Point","coordinates":[2,20]},"value":"2"},
{"id":"3","bbox":[3,30,3,30],"geometry":{"type":"Point","coordinates":[3,30]},"value":"3"},
{"id":"4","bbox":[4,30,4,30],"geometry":{"type":"Point","coordinates":[4,30]},"value":"4"},
{"id":"5","bbox":[5,20,5,20],"geometry":{"type":"Point","coordinates":[5,20]},"value":"5"}
]
}
Another Query:
&bbox=2,30,8,30
{"total_rows":0,"rows":[
{"id":"3","bbox":[3,30,3,30],"geometry":{"type":"Point","coordinates":[3,30]},"value":"3"},
{"id":"4","bbox":[4,30,4,30],"geometry":{"type":"Point","coordinates":[4,30]},"value":"4"}
]
}