I am looking at this.
I have queries that return up to 10,000 rows and usually fills up the memory
I am currently doing:
$this->db->get()->result_array();
Is there a way to not load all of the data into memory and use some sort of cursor? It seems result and result array are both arrays. (one is array of objects other is an array of arrays)
If you are using Active Record, which I would personally recommend, limit() should achieve what you are looking for.
$this->db->limit();
Can also use in this way, slightly easier and less lines of code:
$query = $this->db->get('Table_Name', 50); //syntax db->get('Table',limit_val);
return $query->result();
Also can return a limit with an offset:
$this->db->limit(10, 20); // Second parameter lets you set a result offset
Link for more help on Active Record Query's
https://www.codeigniter.com/userguide2/database/active_record.html#select
Mysql offset is the pointer you are looking for.
You can use it like:
$this->db->get(tableName,10,20);
The second parameter and the third one help you to set limit and offset .
Here are more details
Related
I am using below code to get records with specified condition, and then to update only the same records.
$this->db->where('Parameter1', 'TRUE');
$query = $this->db->get('Messages');
$this->db->where('Parameter1', 'TRUE');
$this->db->set('Parameter1', 'FALSE');
$this->db->update('Messages');
This works, but calling two times the same query using where() command seems like wasting of server power. Is it possible to make get() command not reset query or to use the previous record set in the update command?
I doubt this is something you really need to worry about taking up too many resources, and you can't really reuse the where clause in the actual sql query. But if you'd like you can refactor to get slightly cleaner code.
$unread = array('Parameter1'=>TRUE);
$read = array('Parameter1'=> FALSE);
$query = $this->db->get_where('Messages', $unread);
$this->db->update('Messages', $read, $unread);
Note:
In your code your getting every element where Parameter1 is set to true, and then changing every one of those elements to false. This almost certainly is not desirable, but perhaps it is a problem you take care of somewhere else in your real application.
String q="insert into ChooseRandom (Qid,Question,Option1,Option2,Option3,Option4,Answer,isImage,ImageQuestion) SELECT Qid,Question,Option1,Option2,Option3,Option4,Answer,isImage,ImageQuestion FROM model ORDER BY RAND()";
I am using the above query for randomize sql result set and insert the randomized data's into new table. this works good but this query randomize the rows in table. i need to randomize the rows as well as column and insert into new table.
How can i do this can any one help me to fix this
Thanks in advance
compare to image 1 and image 2 options are randomized as well as rows are randomized.
how can i achieve this
Actually my problem is i can randomize my database table data using rand() function in mysql. but i need to shuffle my column's all so.
i solve my issue but using array list in java i just add the column values to list and than i shuffle the list than i get the list item one by one the code what i used is given bellow .
List<String> lst = new ArrayList<String>();
op1=rs.getString("Option1");
op2=rs.getString("Option2");
op3=rs.getString("Option3");
op4=rs.getString("Option4");
lst.add(op1);
lst.add(op2);
lst.add(op3);
lst.add(op4);
Collections.shuffle(lst);
op1=lst.get(0);
op2=lst.get(1);
op3=lst.get(2);
op4=lst.get(3);
than i used this values to what ever i want.
When I query some view in Couchbase I get the response that has following structure:
{
"total_rows":100,
"rows":[...]
}
'total_rows' is very useful property that I can use for paging.
But lets say I select only a subset of view using 'start_key' and 'end_key' and of course I don't know how big this subset will be. 'total_rows' is still the same number (as I understand it's just total of whole view). Is there any easy way to know how many rows was selected in subset?
You can use the in-built reduce function _count to get the total count of your query.
Just add _count as reduce function for your view. After that, you will need to make two calls to couchbase:
In one call, you'll set the query param reduce=true (along with either group=true or group_level=n, depending upon how you're sending your key(s)). This will give you the total count of your filtered rows.
In the other call, you'll disable the reduce function with reduce=false because you now need the actual rows.
You can find more details about map and reduce at http://docs.couchbase.com/admin/admin/Views/views-writing.html
You can just use an array count/total/length in whatever language you are using.
For example in PHP:
$result = $cb->view("dev_beer", "beer_by_name", array('startkey' => 'O', 'endkey'=>'P'));
echo "total = >>".count($result["rows"])
If you're actually wanting to paginate your data then you should use limit and skip:
http://www.couchbase.com/docs/couchbase-manual-2.0/couchbase-views-writing-querying-pagination.html
If you have to paginate the view in the efficient way, you actually don't need to specify both start and the end.
Generally it is possible to use startkey/startkey_id and limit. In this case the limit will tell you that the page won't be bigger than known size.
Both cases are described in CouchDB book: http://guide.couchdb.org/draft/recipes.html#pagination
Here is how it works:
Request rows_per_page + 1 rows from the view
Display rows_per_page rows, store + 1 row as next_startkey and next_startkey_docid
As page information, keep startkey and next_startkey
Use the next_* values to create the next link, and use the others to create the previous link
Have not been able to find any information on this, I could do this in its own but I feel keeping it in the query might be the best option, if its possible.
Basically I want to try to add a top level "statistics" portion of a query.
So when I get the results I will see it like so
num_rows = 900
distinct_col = 9
results = array()
This way I can loop the results normally, and then pull out information that I would only need once outside of it. Is this possible?
EDIT:
I am not looking for the normal mysql statistics like num_rows exactly. But in a case where lets say you limit the results to ten, num_rows would return 10, but you want total results, so 900. In most cases I would just use another query and look just for the amount, however combining it all into one query logically seems faster for me. There is also more then just the num_rows I may need, say they are all products and have a specific category, I would need to count the amount of categories all items fall under. So looping the raw results when there is only one result for those columns is sillyness.
EDIT 2:
To clarify further I need to get some counts on some columns, and maybe a min-max result on a join. Having it return on every loop would work, but the same exact return uselessly returning on every loop when its only needed once does not seem logical. I am no MySQL expert and am mainly just trying to make sure I come up with the most logical and fastest method to get the required data.
Here is a PHP return example:
array(
[num_rows] => 900,
[categories] => 9,
[min_price] => 400,
[max_price] => 900,
[results] => array(
[0] => //row array
[1] -> //row array
)
);
Mysql returns its default num rows before you "fetch" the results, having custom results added there may be sufficient.
Dunno why do you need that but that's very easy to get
Assuming you are using safeMysql (though you can use whatever way to get data into array)
$results = $db->getAll("SELECT * FROM t");
$num_rows = count($results);
$num_cols = count($results[0]);
that's all
I am mainly just trying to make sure I come up with the most logical and fastest method to get the required data.
Yes, you are.
Nothing wrong with getting aggregated data with every loop.
As for the count beyond LIMITs - when you need it, you can use mysql's SQL_CALC_FOUND_ROWS / FOUND_ROWS() feature
I have the following Array (for example):
array = [234,675,11,233,99]
I than Run something like the following ActiveRecord Query:
User.where(id: array)
Array returned will be randomly ordered.
However, I want the returned results to be based on the order of the original array (i.e. id=234 to be first and id=99 to be last).
I could loop on the array and fire 5 separated queries, is there better way to do so?
Duplicated with:
ActiveRecord.find(array_of_ids), preserving order
So basically the code should be:
User.find(ids).order("field(id, #{ids.join(',')})")