I have a nested repeated structure, the repeated structure is of variable length. For example, it could be a person object with a repeated structure that holds cities the person has lived in. I'd like to find the last item in that list say to find current city person lives in. Is there an easy way to do this, I tried looking around jsonpath functions but I'm not sure how to use it with "within". Any help please?
1) You can use LAST and WITHIN
SELECT
FIRST(cell.value) within record ,
LAST(cell.value) within record
FROM [publicdata:samples.trigrams]
where ngram = "! ! That"
2) or if you want something more advanced you can use POSITION
POSITION(field) - Returns the one-based, sequential position of field within a set of repeated fields.
You can check the samples from trigrams (click on Details to see the unflatten schema)
https://bigquery.cloud.google.com/table/publicdata:samples.trigrams?pli=1
And when you run POSITION, you get the ordering of that field.
SELECT
ngram,
cell.value,
position(cell.volume_count) as pos,
FROM [publicdata:samples.trigrams]
where ngram = "! ! That"
Now that you have the position, you can query for last one.
Related
forgive me for shoddy coding/description of the problem. I'm new to programming and this is my first question!
Anyways, I am building a simple inventory system with tkinter while using mySQL as a database. Currently, I am working on a feature that would allow a user to pick a department using an Optionmenu and then get all the items in that department. I have the items listed in one table and the departments listed in another with a FOREIGN KEY connecting the items table to the primary key (department_string) in the departments table.
My goal is to have mySQL deliver a list of departments and then to have the Optionmenu use that list for it's options. I then need to query the database with the department selected in the Optionmenu to find all the items in that department. My problem is that variable.get() from the Optionmenu returns parentheses and commas that is first received when I query the database the first time. This makes it so I cannot directly input the variable.get() into the string in the cursor. Here is the code:
department_cursor.execute("SELECT department_string FROM departments")
department_list = department_cursor.fetchall()
variable = StringVar(search_window)
variable.set(department_list[0])
user_entry = OptionMenu(search_window, variable, *department_list)
***
cursor_b.execute("SELECT item WHERE department_string = " + "'" + str(variable.get()) + "'")
I believe the problem is that variable.get() provides the special characters like the parentheses and comma that came from the original mySQL query. For example if the departments are HR, Warehouse, R&D then mySQL returns [('HR',), ('Warehouse',), ('R&D',)] this is then fed into the Option menu which then variable.get() spits out something like ('HR',), and so mySQL doesn't recognize this.
So far the only things I can think of is to use a for loop to delete all the special characters in what the Optionmenu returns or to hard code what the string for each department should be. Both seem suboptimal and although I'm pretty new to programming I think it's a little too much like rube goldberg machine.
Anyways, if you made it this far, thank you so much for reading this! Once again, I'm brand new to all of this so any help you can give me is greatly appreciated!
.get() is going to return whatever is in the optionmenu. If the values you put into the optionmenu have the undesirable characters, the value you get out will too.
The database call is going to return a list (rows) of lists (columns), and it doesn't look like you're taking that into account.
I have a model called lists, which has a column called item_ids. item_ids is a JSON column (MySQL) and the column contains array of UUIDs, each referring to one item.
Now when someone creates a new list, I need to search whether there is an existing list with same set of UUIDs, and I want to do this search using query itself for faster response. Also use ActiveRecord querying as much as possible.
How do i achieve this?
item_ids = ["11E85378-CFE8-39F8-89DC-7086913CFD4B", "11E85354-304C-0664-9E81-0A281BE2CA42"]
v = List.new(item_ids: item_ids)
v.save!
Now, how do I check whether a list exists which has item ids exactly matches with that mentioned in query ? Following wont work.
list_count = List.where(item_ids: item_ids).count
Edit 1
List.where("JSON_CONTAINS(item_ids, ?) ", item_ids.to_json).count
This statement works, but it counts even if only one of the item matches. Looking for exact number of items.
Edit 2
List.where("JSON_CONTAINS( item_ids, ?) and JSON_LENGTH(item_ids) = ?", item_ids.to_json, item_ids.size).count
Looks like this is working
You can implement a has many relation between lists and items and then access like this.
List.includes(:item).where('items.id in (?)',item_ids)
To implement has_many relation:
http://guides.rubyonrails.org/association_basics.html#the-has-many-through-association
I have an problem, i am using simple query to select from column (VARCHAR), what user searched and appending +1 string to it from database.
For example if i had in table following values (title):
My name is SOMEONE.
Have a great day.
It's possible.
Mouse on the way.
My mouse is on other way.
Is there any mouse?
And user wants to retreive the keyword mouse.
I wish to output him following from the values above:
Mouse on
mouse is
mouse?
Also if the user types Mouse on it should return
Mouse on the and so on, appending one string from database to found ones (if any).
I could do it outside of database, but i need after retreiving string DISTINCT values with Limitation.
For now i used SUBSTRING_INDEX(title , ' ', ".$totalWords.")
where $totalWords were the user sent to my programme.
Problem with my query was when you type other it will find and return:
My mouse but that's not what i want, i want to it output from found string +1 append so it would be other way
If i wasn't so clear i will give my best to explain it better.
Thank you.
You could use LOCATE(substr,str) 1 to locate the position of the first occurence of the user input. afterwards use SUBSTRING(str FROM pos) 2 to get a substring starting at the position of the first occurence of the user input and finally use your SUBSTRING_INDEX(title , ' ', ".$totalWords.") to get the second word.
PS: Sounds like an autocompletion feature? probably there are better ways to do it than with a relational database
I know how to parse json cells in Open refine, but this one is too tricky for me.
I've used an API to extract the calendar of 4730 AirBNB's rooms, identified by their IDs.
Here is an example of one Json file : https://fr.airbnb.com/api/v2/calendar_months?key=d306zoyjsyarp7ifhu67rjxn52tv0t20¤cy=EUR&locale=fr&listing_id=4212133&month=11&year=2016&count=12&_format=with_conditions
For each ID and each day of the year from now until november 2017, i would like to extract the availability of this rooms (true or false) and its price at this day.
I can't figure out how to parse out these informations. I guess that it implies a series of nested forEach, but i can't find the right way to do this with Open Refine.
I've tried, of course,
forEach(value.parseJson().calendar_months, e, e.days)
The result is an array of arrays of dictionnaries that disrupts me.
Any help would be appreciate. If the operation is too difficult in Open Refine, a solution with R (or Python) would also be fine for me.
Rather than just creating your Project as text, and working with GREL to parse out...
The best way is just select the JSON record part that you want to work with using our visual importer wizard for JSON files and XML files (you can even use a URL pointing to a JSON file as in your example). (A video tutorial shows how here: https://www.youtube.com/watch?v=vUxdB-nl0Bw )
Select the JSON part that contains your records that you want to parse and work with (this can be any repeating part, just select one of them and OpenRefine will extract all the rest)
Limit the amount of data rows that you want to load in during creation, or leave default of all rows.
Click Create Project and now your in Rows mode. However if you think that Records mode might be better suited for context, just import the project again as JSON and then select the next outside area of the content, perhaps a larger array that contains a key field, etc. In the example, the key field would probably be the Date, and why I highlight the whole record for a given date. This way OpenRefine will have Keys for each record and Records mode lets you work with them better than Row mode.
Feel free to take this example and make it better and even more helpful for all , add it to our Wiki section on How to Use
I think you are on the right track. The output of:
forEach(value.parseJson().calendar_months, e, e.days)
is hard to read because OpenRefine and JSON both use square brackets to indicate arrays. What you are getting from this expression is an OR array containing twelve items (one for each month of the year). The items in the OR array are JSON - each one an array of days in the month.
To keep the steps manageable I'd suggest tackling it like this:
First use
forEach(value.parseJson().calendar_months,m,m.days).join("|")
You have to use 'join' because OR can't store OR arrays directly in a cell - it has to be a string.
Then use "Edit Cells->Split multi-valued cells" - this will get you 12 rows per ID, each containing a JSON expression. Now for each ID you have 12 rows in OR
Then use:
forEach(value.parseJson(),d,d).join("|")
This splits the JSON down into the individual days
Then use "Edit Cells->Split multi-valued cells" again to split the details for each day into its own cell.
Using the JSON from example URL above - this gives me 441 rows for the single ID - each contains the JSON describing the availability & price for a single day. At this point you can use the 'fill down' function on the ID column to fill in the ID for each of the rows.
You've now got some pretty easy JSON in each cell - so you can extract availability using
value.parseJson().available
etc.
We want to set up a directory of all the organizations working with us. They are incredibly diverse (government, embassy, private companies, and organizations depending on them ). So, I've resolved to create 2 tables. Table 1 will treat all the organizations equally, i.e. it'll collect all the basic information (name, address, phone number, etc.). Table 2 will establish the hierarchy among all the organizations. For instance, Program for illiterate adults depends on the National Institute for Social Security which depends on the Labor Ministry.
In the Hierarchy table, each column represents a level. So, for the example above, (i)Labor Ministry - Level1(column1), (ii)National Institute for Social Security - Level2(column2), (iii)Program for illiterate adults - Level3(column3).
To attach an organization to an hierarchy, the user needs to go level by level(i.e. column by column). So, there will be at least 3 situations:
If an adequate hierarchy exists for an organization(for instance, level1: US Embassy), that organization can be added (For instance, level2: USAID).--> US Embassy/USAID, and so on.
How about if one or more levels are missing? - then they need to be added
How about if the hierarchy need to be modified? -- not every thing need to be modified.
I do not have any choice but working by level (i.e. column by column). I does not make sense to have all the levels in one form as the user need to navigate hierarchies to find the right one to attach an organization.
Let's say, I have those queries in my repository (just that you get the idea).
Query1
var orgHierarchy = (from orgH in db.Hierarchy
select orgH.Level1).FirstOrDefault;
Query2
var orgHierarchy = (from orgH in db.Hierarchy
select orgH.Level2).FirstOrDefault;
Query3, Query4, etc.
The above queries are the same except for the property queried (level1, level2, level3, etc.)
Question: Is there a general way of writing the above queries in one? So that the user can track an hierarchy level by level to attach an organization.
In other words, not knowing in advance which column to query, I still need to be able to do so depending on some conditions. For instance, an organization X depends on Y. Knowing that Y is somewhere on the 3rd level, I'll go to the 4th level, linking X to Y.
I need to select (not manually) a column with only one query that takes parameters.
=======================
EDIT
As I just said to #Mark Byers, all I want is just to be able to query a column not knowing in advance which one. Check this out:
How about this
Public Hierarchy GetHierarchy(string name)
{
var myHierarchy = from hierarc in db.Hierarchy
where (hierarc.Level1 == name)
select hierarc;
retuen myHierarchy;
}
Above, the query depends on name which is a variable. It mighbe Planning Ministry, Embassy, Local Phone, etc.
Can I write the same query, but this time instead of looking to much a value in the DB, I impose my query to select a particular column.
var myVar = from orgH in db.Hierarchy
where (orgH.Level1 == "Government")
select orgH.where(level == myVariable);
return myVar;
I don't pretend that select orgH.where(level == myVariable) is even close to be valid. But that is what I want: to be able to select a column depending on a variable (i.e. the value is not known in advance like with name).
Thanks for helping
How about using DynamicQueryable?
http://weblogs.asp.net/scottgu/archive/2008/01/07/dynamic-linq-part-1-using-the-linq-dynamic-query-library.aspx
Your database is not normalized so you should start by changing the heirarchy table to, for example:
OrganizationId Parent
1 NULL
2 1
3 1
4 3
To query this you might need to use recursive queries. This is difficult (but not impossible) using LINQ, so you might instead prefer to create a parameterized stored procedure using a recursive CTE and put the query there.