I have an external service that allows me to log users into my website.
To avoid getting kicked out of it for overuse I use a MySQL table on the following form that caches user accesses:
username (STRING) | last access (TIMESTAMP) | has access? (TINYINT(1) - BOOLEAN)
If the user had access on a given time I trust he has access and don't query the service during a day, that is
query_again = !user["last access"].between?(Time.now, 1.day.ago)
This always returns true for some reason, any help with this logic?
In ranges (which you effectively use here), it is generally expected that the lower number is the start and the higher number is the end. Thus, it should work for you if you just switch the condition in your between? call
query_again = !user["last access"].between?(1.day.ago, Time.now)
You can test this yourself easily in IRB:
1.hour.ago.between?(Time.now, 1.day.ago)
# => false
1.hour.ago.between?(1.day.ago, Time.now)
# => true
Related
I have some data in a MySQL database that I'm filter and pulling into R studio to work with. I've come across a weird problem where I can't use elements of a vector with time stamps to filter the data. However, the filter function does work if I hardcode the time stamp of interest or assign the value to a variable first.
Can anyone else reproduce and/or explain this behavior?
The data below will also have a vector of end times and I'd like to filter the data by the intervals defined in the start and end times without using tons of or statements if anyone has a good way to do that.
# vector with times of interest for filtering
start.times <- as.POSIXct(c("2021-06-10 15:30:00", "2021-06-17 12:50:00", "2021-06-18 14:12:00"), tz = "UTC")
# dbconn is a connection to a MySQL database
# table is the name of the first table in the database
# the first column of the table is named time and contains an SQL timestamp in UTC
table <- dbListTables(dbconn)[1]
data.to.filter <- tbl(dbconn, table) # want to filter this based on time column
# this does not work. It returns Error: Operand should contain 1 column(s) [1241]
filter(data.to.filter, time > (start.times[1]))
# This does work
start.time1 = start.times[1]
filter(data.to.filter, time > (start.time1))
# This also works
filter(data.to.filter, time > ("2021-06-10 15:30:00"))
This is likely because of how dbplyr translates R code to SQL code. When you call
filter(data.to.filter, time > (start.times[1]))
The translator will likely identify the transformation time > (start.times[1]) and determine how to translate it. If it translates to SQL before interpreting the contents, then it will translate start.times[1] rather than evaluating it.
You could test this using show_query:
filter(data.to.filter, time > (start.times[1])) %>% show_query()
I would expect you to see [1] turn up in the generated SQL. This would confirm that translation is occurring before evaluation.
If you want to force evaluation before translation, then I suspect you can accomplish this via enquo or eval. I recommend you take a look at Advanced R, Evaluation.
I'm trying to set up an ability to get some numbers from my Sphinx indexes, but not sure how to get the info I want.
I have a mysql db with articles, sphinx index set up for that db and full text search, all working. What I want is to get some numbers:
How many times search text (keyword, or key phrase) appears over all articles for all time (more likely limited to "articles from time interval from X and to Y")
Same as previous but for how many times 2 keywords or keyphrases (so "x AND y") appear in same articles
I was doing something similar to first manually using bat file I made
indexer ind_core -c c:\%SOME_PATH%\development.sphinx.conf --buildstops stats.txt 10000 --buildfreqs
Which generated me a txt with all repeating keywords and how often they appear at early development stages, which helped to form a list of keywords I'm interested in. Now I'm trying to do the same but just for a finite list of predetermined keywords and integrated into my rails project to be able to build charts in future.
I tried running some queries like
#testing = Article.search 'Keyword1 AND Keyword2', :ranker => :wordcount
but I'm not sure how it works and how to process the result, as well as if that's what I'm looking for.
Another approach I tried was manual mysql queries such as
SELECT id,title,WEIGHT() AS w FROM ind_core WHERE MATCH('#title keyword1 | keyword2') OPTION ranker=expr('sum(hit_count)');
but I'm not sure how to process results from here either (as well as how to actually implement it into my existing rails project), and it's limited to 20 lines per query (which I think I can change somewhere in settings?). But at least looking at mysql results what I'm interested in is hit_count over all articles (or all articles from set timeframe).
Any ideas on how to do this?
UPDATE:
Current way I found was to add
#testing = Article.search params[:search], :without => {:is_active => false}, :ranker => :bm25
to controller with some conditions (so it doesn't bug out from nil search). :is_active is my soft delete flag, don't want to search deleted entries, so don't mind it. And in view I simply displayed
<%= #testing.total_entries %>
Which if I understand it correct shows me number of matches sphinx found (so pretty much what I was looking for).
So, to figure out the number of hits per document, you're pretty much on the right track, it's just a matter of getting it into Ruby/Thinking Sphinx.
To get the raw Sphinx results (if you don't need the ActiveRecord objects):
search = Article.search "foo",
:ranker => "expr('SUM(hit_count)')",
:select => "*, weight()",
:middleware => ThinkingSphinx::Middlewares::RAW_ONLY
… this will return an array of hashes, and you can use the weight() string key for the hit count, and the sphinx_internal_id string key for the model's primary key (id is Sphinx's own primary key, which isn't so useful).
Or, if you want to use the ActiveRecord objects, Thinking Sphinx has the ability to wrap each search result in a helper object which passes appropriate methods through to the underlying model instances, but lets weight respond with the values from Sphinx:
search = Article.search "foo",
:ranker => "expr('SUM(hit_count)')",
:select => "*, weight()"; ""
search.context[:panes] << ThinkingSphinx::Panes::WeightPane
search.each do |article|
puts article.weight
end
Keep in mind that panes must be added before the search is evaluated, so if you're testing this in a Rails console, you'll want to avoid letting the console inspect the search variable (which I usually do by adding ; "" at the end of the initial search call.
In both of these cases, as you've noted, the search results are paginated - you can use the :page option to determine which page of results you want, and :per_page to determine the number of records returned in each request. There is a standard limit of 1000 results overall, but that can be altered using the max_matches setting.
Now, if you want the number of times the keywords appear across all Sphinx records, then the best way to do that while also taking advantage of Thinking Sphinx's search options, is to get the raw results of an aggregate SUM - similar to the first option above.
search = Article.search "foo",
:ranker => "expr('SUM(hit_count)')",
:select => "SUM(weight()) AS count",
:middleware => ThinkingSphinx::Middlewares::RAW_ONLY
search.first["count"]
I have some reporting methods throughout my app and in some cases I want to return the count (for a dashboard), and in others, return the full result set for viewing the details of a report.
What I'm wondering, is there a way to dynamically choose to show the count (instead of what I'm doing here):
def get_query_results(reporting_parameters, count_only = true)
#put together reporting details...
if count_only
MyModel.where(query).count
else
MyModel.where(query)
end
end
I considered setting a local variable to the result of my query parameter, and then call count, but that queries the database again (and even if it didn't it could increase memory usage).
Is there a way to do an effective way to do this in one query? This is one of several queries I have like this in my app, otherwise I wouldn't care. Also, I'd use a ternary, but the actual query conditions in my app are much longer than my example here and it makes it unreadable.
Suppose you are doing this:
#collection = get_query_results(...)
Then you can do this afterwards instead of inside of the action:
#collection.count
And if you like to call another method:
def total_number(collection)
collection.count
end
#collection = get_query_results(...)
no_of_records = total_number(#collection)
I have this column_property which checks how many credits a user is allowed:
select(
[func.ifnull( func.sum( orders_table.c.quantity ), 0 )],
orders_table.c.user_id == users_table.c.id
).where( and_(
orders_table.c.date_added < now_unix(),
orders_table.c.date_expires > now_unix(),
orders_table.c.status == STATUS_COMPLETED
) ).label( 'userAllowedCredits' ),
deferred = True
The now_unix() method returns the current unix timestamp but the problem is that this method is loaded only once and every time I call this userAllowedCredits property the query searches based on the same initial value that was saved when my app was started. I need this now_unix() method to return the actual current timestamp at every call.
Do I make any sense?
You probably store this expression at startup, so now_unix() is only executed at that time. From the SqlAlchemys point of view, it's just a value. How to solve this, depends a bit on your use case. You could use now(), which would use the database build in now() function. If you have to use your own now_unix(), you can still pass it as a parameter.
I've got a really, really odd problem manifesting on a big Rails e-commerce app and thought I'd see if anyone has good insight. I have an"Order" model with many associations. If I create a new instance, and then set one particular column value and "save!" the "save!" is succeeding without errors, but the change isn't actually persisted to the DB. I'll run through the scenario below:
#order = Order.create!(<some attributes>)
=> true
#order.shipping_method_id
=> 1
#order.shipping_method_id = 203
=> 203
#order.save!
=> true
#order.shipping_method_id
=> 1
To try and debug this I actually prepended a before_save filter and I can see that when this first filter is called after setting the value, it is correct ("203") BUT the very next before_save after the 6-or-so built-in "autosave_foo_bar_quux" filters (for nested associations) it is back to "1".
Oddly, if I just reload the order (#order.reload), change the column value and save! the update does succeed.
In both cases, doing #order.changed shows that ActiveModel recognizes the column value change for shipping_method_id. In the first, though, the SQL logging shows that the order row is not updated.
I feel like I'm going insane. Any ideas? Also, let me know if there's anything else I can post here for more context.