I will store, in couchbase, something like this:
key: foo
value: {
some_id => bar,
/* other fields */
}
for me, is easy find the value, using the key 'foo', but how I can find the key 'foo' using the 'some_id' bar (and it can returns more than one result)?
I was reading about views + where clause, it is the only way?
Besides Couchbase views (as you mentioned), as of Couchbase version 3.x you will have N1QL query language. You can specify "where" condition to select your json objects without having any views in place.
So as per your example, you should be able issue a query like that:
SELECT *
FROM your_bucket_name
WHERE some_id = 'bar'
Try out N1QL tutorial
Another way could be to utilize Couchbase integration with ElasticSearch and perform search query in ES engine that will return you all keys it found based on your search criteria.
Related
I'm working with Yii2 Relational Database / Active Query Models and I ran into an issue trying to use the magic method getModelName() with $this->hasMany()->viaTable() to set the relation while trying to sort by a sort column in my junction table.
First I tried to just add my ->orderBy() clause to the main query:
return $this->hasMany(Category::class,
['id' => 'categoryId'])
->viaTable('{{kits_x_categories}}',
['kitId' => 'id'])
->orderBy('{{kits_x_categories}}.sort asc');
That didn't work as expected and upon further digging I found out that this results in two separate queries, the first one selects my category Ids into an array, then uses said array for a WHERE IN() clause in the main (2nd) query to get the actual models that are related.
My first thought was to use the 3rd function($query) {} callback parameter of the ->viaTable() call and putting my $query->orderBy() clause on there:
return $this->hasMany(Category::class,
['id' => 'categoryId'])
->viaTable('{{kits_x_categories}}',
['kitId' => 'id'],
function($query) {
return $query->orderBy('{{kits_x_categories}}.sort asc');
}
);
However, all that did was return the category ID's in my desired order but ultimately had no effect on the main query that does the IN() condition with said ids since the order of the ids in the IN() condition have no effect on anything.
Finally, I ended up with this which lets it do what it wants, but then forces in my join to the main query with the IN() condition so that I can have the main query sort by my junction table sort column. This works as desired:
return $this->hasMany(Category::class,
['id' => 'categoryId'])
->viaTable('{{kits_x_categories}}',
['kitId' => 'id'])
->leftJoin('{{kits_x_categories}}', '{{kits_x_categories}}.categoryId = {{categories}}.id')
->where(['{{kits_x_categories}}.kitId' => $this->id])
->orderBy('{{kits_x_categories}}.sort asc');
This results in 2 queries.
First the query gets the category ids from the join table:
SELECT * FROM `kits_x_categories` WHERE `kitId`='49';
Then the main query with the IN() condition and my forced join for sort:
SELECT `categories`.* FROM `categories`
LEFT JOIN `kits_x_categories` ON `kits_x_categories`.categoryId = `categories`.id
WHERE (`kits_x_categories`.`kitId`='49') AND (`categories`.`id` IN ('11', '7', '9'))
ORDER BY `kits_x_categories`.`sort`
So here is my actual question... This seems largely inefficient to me but I am by no means a database/SQL god so maybe I just don't understand fully. What I want is to understand.
Why does Yii do this? What is the point of making one query to get the IDs first, then making another query to get the objects based on the ids of the relation? Wouldn't it be more efficient to just do a regular join here? Then, in my opinion, sorting by a junction sort column would be intuitive rather than counter-intuitive.
The only thing I can think of is has to do with the lazy vs eager loading of data, maybe in lazy in only gets the IDs first then when it needs to load the data it pulls the actual data using IN()? If I used joinWith() instead of viaTable() would that make any difference here? I didn't dig into this as I literally just thought of that as I was typing this.
Lastly, In this scenario, There is only going to be a few categories for each kit so efficiency is not a big deal but I'm curious are there any performance implications in my working solution if I were to use this in the future on a different model set that could have thousands+ of relations?
Yii 2 does that:
To support lazy loading.
To support cross-database relations such as MySQL -> Redis.
To reduce number of edge-cases significantly so internal AR code becomes less complicated.
3rd party software is usually designed to get you started with databases. But then they fall apart when the app grows. This means that you need to learn the details of the underlying database in addition to the details of the layer.
Possibly this specific issue can be solved by improving the indexes on the many-to-many table with the tips here: http://mysql.rjweb.org/doc.php/index_cookbook_mysql#many_to_many_mapping_table
This, of course, depends on whether the layer lets you tweak the schema that it created for you.
If there is a way to write "raw" SQL, that might let you get rid of the 2-step process, but you still need to improve the indexes on that table.
I'm trying to optimise a large MySQL query. I accidentally found out that a query with all fields listed (SELECT Orders.id AS Orders__id, <...>; the default CakePHP behaviour) takes 4 times longer compared to a query with just SELECT * FROM - 0.324 seconds vs. 0.084 seconds; checked several times.
I'm wondering if I can disable this behaviour. I've tried:
adding 'fields' => '*' to the find() options or calling ->select('*'), but it results in SELECT Orders.* AS Orders__* which throws an SQLSTATE[42000] error.
getting rid of the aliased title with ->select(['*' => '*']) as per query-builder.html#selecting-data, but that results in SELECT * AS * which also throws an error
using ->enableAutoFields(false)
I also tried to Google but I don't even know how to call this
It seems you want to override the benefits of the ORM so I'm going to suggest a method that should not be used in normal operations.
$datasource = ConnectionManager::get('default');
$datasource->execute('SELECT * FROM some_table;');
If you want to hydrate entities you must allow the select statement to alias the fields so the above will not give you entities.
My opinion is that you should use the normal select and optimize your caching strategies for this data.
Apparently, I can't not have the fields listed because of how CakePHP ORM was designed.
So instead, solved by manually whitelisting only the fields I actually need. The query is still relatively fast after all (around 100 ms. according to my measurements).
I am currently working on a project where I have to query data from different devices (example data attached below). Originally I was querying this data using
Select * From `table` Where `data` Like '%5.3%'
This took a long time for every query so I created a FULLTEXT index on data. I then wrote the following query to find the same product version, but it does not get all the same information the LIKE query gets.
Select `data` From `table`
Where match(`data`)against('+"5.3"' IN BOOLEAN MODE);
I was wondering what the fastest way to search this type of data is. Should I modify the index I created or is there another method to do something like this. I have attached sample data where I try to find the version number 5.3 below. Thanks for your time
SSH-2.0-OpenSSH_5.3p1 Debian-5ubuntu1.3
Key type: ssh-rsa
Key: AAAwtywtyhwhtgwJi40rvS84YLx5a19Psfdgr45ywrwsrH6msVLSwertwthgwmT
KnbqSuozljiCcdl/M/M/dyc+ivekkEsrKSPaPRLF46XeM
TfOZHfQKs7MYS1Zu4Ed/eB2hVPAUVQgsadfk9ReFr0QO985n+XJUQv55WZE8loZ4lQV+/G/5
Fingerprint: 00:00:00:00:00:00:00
I am new to Couchbase noSql database.
I am trying to create a view, i want this view should give result as below SQL query.
SELECT * FROM Employee e WHERE e.name = "DESM%" AND e.salary < 1000 ORDER BY e.id desc
Any suggestion is very appreciated.
If you look at existing beer samples in Couchbase (it comes with it), you will find views defined there. In admin console you can run a view. Notice when you run a view you can provide filtering criteria and sort order for the result...that might be an equivalent for your SQL like functionality. Read more on Views and Indexes
yet another option is to use Couchbase v3 that comes with its own N1QL query language that can serve as another alternative. You can try it out online here.
I'm currently practicing using Sphinx, I've not far off done much, except the configuration what I'm trying to do. The sql_query key is leaving me somewhat confused what to put there, I read in the Sphinx documentation of sql_query but it doesn't seem to clear my mind from knowing what to do since I have many SELECTs in my web application, and I want to use Sphinx for my search and the SQL is often changed (upon user search filtering).
As of my search using MySQL, I want to integrate Sphinx to my web application, if the sql_key is not optional, do I have to expect to put the whole search SQL query into that field or do I pick out the necessary fields from tables to start a reindex?
Can someone point me to the right direction so I can get things going well with Sphinx and my web application.
sql_query is mandatory , it's run by sphinx to get the data you want to be indexed from mysql . You can have joins , conditions etc. , must be a valid sql query . You should have something like "SELECT id ,field1,field2,fieldx from table" . id must be a primary id .Each row returned by this query is considered a document ( which is returned by sphinx when you search ) .
If you have multiple tables ( that are very different by meaning - users , articles etc.) - you need to create an index for each .
Read tutorials from here : http://sphinxsearch.com/info/articles/ to understand how sphinx works .
You can create a sql query to get union set of records from the Database. If you do multiple table joining and query to select the best result set, you can do it with Sphinx too.
You may run into a few trouble with your existing table structure in the database.
Like :
Base table does not have integer primary key field
Create a new table which has two fields. One for the integer id field and the other field to hold the primary key of the base table. Do an inner join with that table and select the id field from that table.
Eg. SELECT t1.id, t2.name, t2.description, t2.content FROM table_new t1 INNER JOIN table_2 t2 WHERE t1.document_id = t1.thread_id INNER JOIN REST_OF_YOUR_SELECT_QUERY
The ta.id is for Sphinx search engine to do its internal indexing.
You filter data by placing WHERE clause and filtering
You can do that in Sphinx by setting filters dynamically based on the conditions.
You select and join different tables to get results
This also can be done by setting different sources and indexes based on your requirements.
Hope this would help you to get an understanding what you need to add and modify to start thinking how Sphinx search engine can be configured to your requirements. Just come here again if your need more help.