How to deal with Parsing Object from Join Query - mysql

I'm currently working on my first API with the Perfect framework. It's been a while since I made an API myself so I must admit my SQL and API logic is a little rusty.
I'm using a MySQL database for my implementation.
For sake of example I'll explain my database structure below;
I have a table which resembles an Object, let's call this Table A. Table A has a Varchar based id as primary key.
There are 2 other tables let's call them Table B and Table C. Table A has a one to many relation to both Table B and C. Where the id of table A is the foreign key.
What I'm trying to do is obtain everything with one query and cast it to an object in my backend.
By using outer joins I'm making the call to retrieve all the required data.
SELECT control.id, control.type, control.description, control.address, control.city, control.created, control.updated, control.latitude, control.longitude, images.id AS image_id, images.image, images.description AS image_description, updates.id AS update_id, updates.still_present, updates.created_at AS update_created
FROM Control control left outer join control_images images
ON control.id=images.control_id
left outer join Control_Updates updates
ON control.id=updates.control_id
Now is my question what would be the best way to store this data in an object that holds an array of updates and an array of images.
Before writing the join query I only attempted to get the values from Table A I used the following code to cast the results to my desired object.
let result = mysql.storeResults()
let checkResult = self.checkResult(result: result, response: response)
response = checkResult.response
var controls: [Control] = []
while let row = result?.next() {
let type = Types(rawValue: row[1].unwrap)!
let control = Control(id: row[0].unwrap, type: type, description: row[2].unwrap, address: row[3].unwrap, city: row[4].unwrap, latitude: Double(row[7].unwrap).unwrap, longitude: Double(row[8].unwrap).unwrap)
controls.append(control)
}
obviously this will just return duplicate objects apart from the images and updates of course.
I'm wondering if this is the best way to do it or if I should call a new query in the while loop

The best way to resolve this issue, by still only using one query and one loop is by using 'hashmaps'. I'm not familiar with Perfect framework, but in PHP it would look something like:
// Get results from the db:
$results = $db->execute($query, $params);
// Define map for controls:
$map = [];
// Loop over results/rows
foreach($results as $row){
// Get unique identifier for the Control model:
$controlId = $row['id'];
// Check if control is NOT already in map:
if(!isset($map[$controlId]){
// Add control to map:
$control = [
'id' => $controlId,
'description' => $row['description'],
'images' => []
// other fields
];
// Add control to map:
$map[$controlId] = $control;
}
else{
// Control exists, retrieve it from the map:
$control = $map[$controlId];
}
// Retrieve unique identifier of the image:
$imageId = $row['image_id'];
// Same tactic with hasmap, check if control already has the image, if not add it
if(!isset($control['images'][$imageId]){
// add the image to the hashmap:
}
else{
// Image is already added, the content from the 'update' data is not added yet, handle that part (also with a hash map)
}
}
Hope that helps you figure it out in Perfect framework

Related

Multi-parameter search with mysql and node.js

Let me preface by saying I'm very new to SQL (and back end design) in general. So for those annoyed with noob questions, please be gentle.
BACKGROUND:
I'm trying to build a product test database (storing test data for all our products) where I want a user to be able to refine a search to find test data they actually want. For example, they may start by searching for all products of a certain brand name, and then refine it with a product type, and/or refine it with a date range of when the test was done.
PROBLEM:
I'm having a hard time finding information on how to implement multi-parameter searches with mysql and node.js. I know you can do nested queries and joins and such within pure SQL syntax, but it's not abundantly clear to me how I would do this from node.js, especially when certain search criteria aren't guaranteed to be used.
Ex:
CREATE PROCEDURE `procedureName`(
IN brandname VARCHAR(20),
producttype VARCHAR(30))
BEGIN
SELECT * FROM products
WHERE brand = brandname
AND product_type = producttype;
END
I know how to pass data from node.js to this procedure, but what if the user didn't specify a product type? Is there a way to nullify this part of the query? Something like:
AND product_type = ALL;
WHAT I'VE TRIED:
I've also looked into nesting multiple SQL procedures, but passing in dynamic data to the "FROM" clause doesn't seem to be possible. Ex: if I had a brandname procedure, and a product type procedure, I don't know how/if I can pass the results from one procedure to the "FROM" clause of the other to actually refine the search.
One idea was to create tables with the results in each of these procedures, and pass those new table names to subsequent procedures, but that strikes me as an inefficient way to do this (Am I wrong? Is this a completely legit way to do this?).
I'm also looking into building a query string on the node side that would intelligently decide what search criteria have been specified by the front end, and figure out where to put SQL AND's and JOIN's and what-nots. The example below actually works, but this seems like it could get ugly quick as I add more search criteria, along with JOINS to other tables.
// Build a SQL query based on the parameters in a request URL
// Example request URL: http://localhost:3000/search?brand=brandName&type=productType
function qParams(req) {
let q = "SELECT * FROM products WHERE ";
let insert = [];
if(req.query.brand) {
brandname = req.query.brand; // get brandname from url request
q = q + `brand = ?`, // Build brandname part of WHERE clause
insert.push(brandname); // Add brandname to insert array to be used with query.
};
if(req.query.type) {
productType = req.query.type; // get product type from url request
insert.length > 0 ? q = q + ' AND ' : q = q; // Decide if this is the first search criteria, add AND if not.
q = q + 'product_type = ?'; // Add product_type to WHERE clause
insert.push(productType); // Add product_type variable to insert array.
}
// Return query string and variable insert array
return {
q: q,
insert: insert
};
};
// Send Query
async function qSend(req, res) {
const results = await qParams(req); // Call above function, wait for results
// Send query string and variables to MySQL, send response to browser.
con.query(results.q, results.insert, (err, rows) => {
if(err) throw err;
res.send(rows);
res.end;
})
};
// Handle GET request
router.use('/search', qSend);
CONCISE QUESTIONS:
Can I build 1 SQL procedure with all my search criteria as variables, and nullify those variables from node.js if certain criteria aren't used?
Is there way to nest multiple MySQL procedures so I can pick the procedures applicable to the search criteria?
Is creating tables of results in a procedure, and passing those new table names to other procedures a reasonable way to do that?
Building the query from scratch in node is working, but it seems bloated. Is there a better way to do this?
Googling "multi-parameter search mysql nodejs" is not producing useful results for my question, i.e. I'm not asking the right question. What is the right question? What do I need to be researching?
One option is to use coalesce():
SELECT p.*
FROM products p
WHERE
p.brand = COALESCE(:brandname, p.brand)
AND p.product_type = COALESCE(:producttype, p.producttype);
It may be more efficient do explicit null checks on the parameters:
SELECT p.*
FROM products p
WHERE
(:brandname IS NULL OR p.brand = :brandname)
AND (:producttype IS NULL OR p.product_type = :producttype);

How to maintain relationships when importing data into Parse.com

I have a database that I'd like to import into Parse.com but having trouble wrapping my head around how the data should be stored in Parse and maintain relationship between this data.
Here's my current data and their relationship.
Table 1: Videos
-> VideoName
-> URL
Table 2: Performers
-> PerformerName
-> Bio
Table 3: Games
-> GameName
-> GameDetail
Table 4: Seasons
-> SeasonName
-> SeasonDetail
The relationships are:
-> Each Season can have more than one video, but each video can only be associated with one season.
-> Each video can have one or more Game
-> Each Video can have one or more Performer
-> Each Game can be associated with one or more Video
-> Each Game can be associated with one or more Performer
-> Each Performer can be in one or more Videos
-> Each Performer can be associated with one or more Game.
This means that there's many to many relationships between the data.
If someone can help me get my thinking straight on how Parse does relational databases/information I'd appreciate it. I have already read the Relational Data documentation but I'm not fully understanding how it applies to my data.
I would like to be able to export the data out of my database and import it into Parse. I am open to any suggestion on getting this data into Parse and maintaining relationships.
I need the relationships because the app is going to be pulling queries like:
-> Retrieve all videos where performer = x
-> Retrieve all games where performer = x
-> Retrieve all performers where game = x
-> Retrieve all videos where season = x
plus others as needed.
I hope someone can help. If you can provide specific examples based on the information I have above that would be excellent! :-)
Parse supports relationships in a number of ways:
Pointer (think foreign key ID)
Array of Pointers (think join table, but only when you always want to get all of them)
Relation (think join table, where you want to query for a sub-set)
What you use depends greatly on your queries. In your case I would suggest the following:
Videos:
Name (string)
URL (string)
Season (pointer)
Games (array, assuming this is a short list)
Performers (array)
Performers:
Name (string)
Bio (string, possibly with HTML or other markup)
Games:
Name (string)
Detail (string)
Performers (array)
Seasons:
Name (string)
Detail (string)
For your queries (in JavaScript since you didn't specify a language):
// setup bits
var Video = Parse.Object.extend("Video");
var Performer = Parse.Object.extend("Performer");
var Game = Parse.Object.extend("Game");
var Season = Parse.Object.extend("Season");
// All videos where performer = x
var videosForPerformerQuery = new Parse.Query(Video);
// assumes performerX has been retrieved or initialised with an ID
// equalTo in this case will find records where performerX is in the array of performers
videosForPerformerQuery.equalTo('performers', performerX);
// if you need to output the full list of performers:
videosForPerformerQuery.include('performers');
// if you need the full season object:
videosForPerformerQuery.include('season');
videosForPerformerQuery.find({
success: function(results) {
// TODO: handle array of video objects in results
},
error: function(error) {
// TODO: handle error
}
});
// All games where performer = x
var gamesForPerformerQuery = new Parse.Query(Game);
// .. identical to above ..
// All performers where game = x
var gameQuery = new Parse.Query(Game);
gameQuery.include('performers');
// gameObjectID is a string key, e.g. "xWMyZ4YEGZ"
gameQuery.get(gameObjectId, {
success: function(game) {
// TODO: game.performers array should be populated, use it
},
// .. error handler too ..
});
// All videos where season = x
var videosForSeason = new Parse.Query(Video);
videosForSeason.equalTo('season', seasonX);
// .. find same as above examples ..
Sometimes you'll want bi-directional joins. You can either manually populate an Array/Relation on each end, or use a new object to hold the relations (especially if the relation contains extra info). Example:
GamePerformers:
Game (pointer)
Performer (pointer)
Role (string)
Query it and include the Game/Performer as needed:
var GamePerformers = Parse.Object.extend("GamePerformers");
var performersForGameQuery = new Parse.Query(GamePerformers);
performersForGameQuery.equalTo('game', gameX);
performersForGameQuery.include('performer');
performersForGameQuery.find({
success: function(results) {
for (var i = 0; i < results.length; i++) {
var object = results[i];
console.log('Performer: '
+ object.get('performer').get('name')
+ ' played role: '
+ object.get('role'));
}
}
});

How to use joins in HTML5 Indexed Db?

I do want to join two table in HTML5 indexed db.
I found lot samples to add, Update, Delete and listing record but can't found any samples to join multiple table.
Sample URL:
*http://users.telenet.be/kristofdegrave/*
Was wondering the same question when just generally looking at these technologies. Did not check your particular example, but here is an example from Mozilla on how to do a join if you need it:
https://hacks.mozilla.org/2010/06/comparing-indexeddb-and-webdatabase/
It is quite a lot of code compared to SQL, but I think the idea is that abstracting these using JavaScript later is quite easy.
There is no join, but you can open multiple object stores and join during cursor iterations.
Using my own ydn-db library, you can query SELECT * FROM Supplier, Part WHERE Supplier.CITY = Part.CITY by
var iter_supplier = new ydn.db.IndexValueIterator('Supplier', 'CITY');
var iter_part = new ydn.db.IndexValueIterator('Part', 'CITY');
var req = db.scan(function(keys, values) {
var SID = keys[0];
var PID = keys[1];
console.log(SID, PID);
if (!SID || !PID) {
return []; // done
}
var cmp = ydn.db.cmp(SID, PID); // compare keys
if (cmp == 0) {
console.log(values[0], values[1]);
return [true, true]; // advance both
} else if (cmp == 1) {
return [undefined, SID]; // jump PID cursor to match SID
} else {
return [PID, undefined]; // jump SID cursor to match PID
}
}, [iter_supplier, iter_part]);
See more detail on Join query article.
As far as I know, IndexedDB does not have an API for doing JOINs, yet. The solutions I adopted involved opening a cursor, looping through the results and doing the join manually. It's the dreadful RBAR approach, but I couldn't find a better solution.
Instead of trying to join, redesign how you store the data so that a join is not required. You do not need to follow the same normalization constraints as SQL when you are using a no-sql approach. No-sql supports storing data redundantly when it is appropriate.
IndexedDB is in fact an object store and normaly in an object store there is less need for joins because you can just save the nested structure.
In the case you show, data from a codetable gets joined with real data. The case to solve this without a join is just fetch your code table into memory (normally this data never changes) and make the join in code.

Populate JOIN into a list in one database query

I am trying to get the records from the 'many' table of a one-to-many relationship and add them as a list to the relevant record from the 'one' table.
I am also trying to do this in a single database request.
Code derived from Linq to Sql - Populate JOIN result into a List almost achieves the intended result, but makes one database request per entry in the 'one' table which is unacceptable. That failing code is here:
var res = from variable in _dc.GetTable<VARIABLE>()
select new { x = variable, y = variable.VARIABLE_VALUEs };
However if I do a similar query but loop through all the results, then only a single database request is made. This code achieves all goals:
var res = from variable in _dc.GetTable<VARIABLE>()
select variable;
List<GDO.Variable> output = new List<GDO.Variable>();
foreach (var v2 in res)
{
List<GDO.VariableValue> values = new List<GDO.VariableValue>();
foreach (var vv in v2.VARIABLE_VALUEs)
{
values.Add(VariableValue.EntityToGDO(vv));
}
output.Add(EntityToGDO(v2));
output[output.Count - 1].VariableValues = values;
}
However the latter code is ugly as hell, and it really feels like something that should be do-able in a single linq query.
So, how can this be done in a single linq query that makes only a single database query?
In both cases the table is set to preload using the following code:
_dc = _db.CreateLinqDataContext();
var loadOptions = new DataLoadOptions();
loadOptions.LoadWith<VARIABLE>(v => v.VARIABLE_VALUEs);
_dc.LoadOptions = loadOptions;
I am using .NET 3.5, and the database back-end was generated using SqlMetal.
This link may help
http://msdn.microsoft.com/en-us/vcsharp/aa336746.aspx
Look under join operators. You'll probably have to change from using extension syntax other syntax too. Like this,
var = from obj in dc.Table
from obj2 in dc.Table2
where condition
select

LINQ variable to list of string without using column names?

In an C# ASP.Net MVC project, I'm trying to make a List<string> from a LINQ variable.
Now this might be a pretty basic thing, but I just cannot get that to work without using the actual column names for the data in that variable. The thing is that in the interests of trying to make the program as dynamic as possible, I'm leaving it up to a stored procedure to get the data out. There can be any amount of any which way named columns depending on where the data is fetched from. All I care about is taking all of their values into a List<string>, so that I can compare user-input values with them in program.
Pointing to the columns by their names in the code means I'd have to make dozens of overloaded methods that all just basically do the same thing. Below is false non-functioning code. But it should open up the idea of what I mean.
// call for stored procedure
var courses = db.spFetchCourseInformation().ToList();
// if the data fails a check on a single row, it will not pass the check
bool passed = true;
foreach (var i in courses)
{
// each row should be cast into a list of string, which can then be validated
// on a row-by-row basis
List courseRow = new List();
courseRow = courses[i]; // yes, obviously this is wrong syntax
int matches = 0;
foreach (string k in courseRow)
{
if (validator.checkMatch(courseRow[k].ToString()))
{
matches++;
}
}
if (matches == 0)
{
passed = false;
break;
}
}
Now below is an example of how I currently have to do it because I need to use the names for the columns
for (int i = 0; i < courses.Count; i++)
{
int matches = 0;
if (validator.checkMatch(courses[i].Name))
matches++;
if (validator.checkMatch(courses[i].RandomOtherColumn))
matches++;
if (validator.checkMatch(courses[i].RandomThirdColumn))
matches++;
if (validator.checkMatch(courses[i].RandomFourthColumn))
matches++;
/* etc...
* etc...
* you get the point
* and one of these for each and every possible variation from the stored procedure, NOT good practice
* */
Thanks for help!
I'm not 100% sure what problem you are trying to solve (matching user data to a particular record in the DB?), but I'm pretty sure you're going about this in slightly the wrong fashion by putting the data in a List. I
t should be possible to get your user input in an IDictionary with the key being used for the column name, and the object as the input data field.
Then when you get the data from the SP, you can get the data back in a DataReader (a la http://msmvps.com/blogs/deborahk/archive/2009/07/09/dal-access-a-datareader-using-a-stored-procedure.aspx).
DataReaders are indexed on column name, so if you run through the keys in the input data IDictionary, you can check the DataReader to see if it has matching data.
using (SqlDataReader reader = Dac.ExecuteDataReader("CustomerRetrieveAll", null))
{
while (reader.Read())
{
foreach(var key in userInputDictionary.AllKeys)
{
var data = reader[key];
if (data != userInputDictionary[key]) continue;
}
}
}
Still not sure about the problem you are solving but, I hope this helps!
A little creative reflection should do the trick.
var courses = db.spFetchCourseInformation()
var values = courses.SelectMany(c => c.GetType().GetProperties() // gets the properties for your object
.Select(property => property.GetValue(c, null))); // gets the value of each property
List<string> stringValues = new List<string>(
values.Select(v => v == null ? string.Empty : v.ToString()) // some of those values will likely be null
.Distinct()); // remove duplicates