I've looked through all of the Simperium API docs for all of the different programming languages and can't seem to find this. Is there any documentation for the data returned from an ".all" call (e.g. api.todo.all(:cv=>nil, :data=>false, :username=>false, :most_recent=>false, :timeout=>nil) )?
For example, this is some data returned:
{"ccid"=>"10101010101010101010101010110101010",
"o"=>"M",
"cv"=>"232323232323232323232323232",
"clientid"=>"ab-123123123123123123123123",
"v"=>{
"date"=>{"o"=>"+", "v"=>"2015-08-20T00:00:00-07:00"},
"calendar"=>{"o"=>"+", "v"=>false},
"desc"=>{"o"=>"+", "v"=>"<p>test</p>\r\n"},
"location"=>{"o"=>"+", "v"=>"Los Angeles"},
"id"=>{"o"=>"+", "v"=>43}
},
"ev"=>1,
"id"=>"abababababababababababababab/10101010101010101010101010110101010"}
I can figure out some of it just from context or from the name of the key but a lot of it is guesswork and trial and error. The one that concerns me is the value returned for the "o" key. I assume that a value of "M" is modify and a value of "+" is add. I've also run into "-" for delete and just recently discovered that there is also a "! '-'" which is also a delete but don't know what else it signifies. What other values can be returned in the "o" key? Are there other keys/values that can be returned but are rare? Is there documentation that details what can be returned (that would be the most helpful)?
If it matters, I am using the Ruby API but I think this is a question that, if answered, can be helpful for all APIs.
The response you are seeing is a list of all of the changes which have occurred in the given bucket since some point in its history. In the case where cv is blank, it tries to get the full history.
You can find some of the details in the protocol documentation though it's incomplete and focused on the WebSocket message syntax (the operations are the same however as with the HTTP API).
The information provided by the v parameter is the result of applying the JSON-diff algorithm to the data between changes. With this diff information you can reconstruct the data at any given version as the changes stream in.
Related
I'm making a Community Connector with the following fields, among others: Age, gender and impressions.
When I try to do a bar chart with Impressions as a metric, Age as a dimension and Gender as a breakdown dimension (or Gender and Age, inverted) I get the following error:
User Configuration Error
This data source was improperly configured.
Invalid argument type.
Error ID: b44d6288
Debugging, I found the problem is that it isn't making a single request to getData() including the three fields (which, when processed, would make the right call to the API and get the right data). Instead it only requests the pair dimension-metric on one request and sometimes also breakdown dimension-metric on others (and, sometimes, dimension-metric with filter info), which gives it a "broken" data which it apparently can't make sense of. As the request to my getData() only includes two fields, I return two fields per row. Info about the third one can't be found nowhere, especially on the request parameter, as far as I can see.
This behavior appeared somewhere along the development of the connector -- at some points this exact combination worked normal.
As this behavior doesn't include code it's really scaring me. Any idea would be deeply appreciated.
I am having difficulty accessing all the data returned by my forms in my post function. I notice a significant discrepancy between what is displayed when I print request.POST vs. when my code accesses this data. Hopefully someone can explain this to me.
Output of print(request.POST):
print(request.POST)
<QueryDict: {'csrfmiddlewaretoken': ['AXMPO...'],
'start_date': ['2019-03-01'], 'end_date': ['2019-03-26'],
'reports': ['4', '1']}>
In order to examine the data my code is dealing with I used the json module to view the data. The behavior of my code during debugging conforms to this representation:
json.dumps(request.POST)
'{"csrfmiddlewaretoken": "AXMPO...",
"start_date": "2019-03-01", "end_date": "2019-03-26",
"reports": "1"}'
It all looks pretty similar until you see the "reports" value. The user selects these reports via an MultipleSelect widget on my form and my code is iterating through the id numbers provided. However, no matter how many reports I select I only get one ID. If anyone can explain why this is happening I would sincerely appreciate it.
Turns out this is a really old school issue. I could wish this was more prominent in the documentation though. The explanation by Simon Willson is below:
"""
This is a feature, not a bug. If you want a list of values for a key, use the following:
values = request.POST.getlist('key')
The reasoning behind this is that an API method should consistently return either a string or a list, but never both. The common case in web applications is for a form key to be associated with a single value, so that's what the [] syntax does. getlist() is there for the occasions (like yours) when you intend to use a key multiple times for a single value.
""" - Simon Willson, 13 years ago.
I am using ELKI to cluster data from CSV file
I use
-resulthandler ResultWriter
-out folder/
to save the outputdata
But as an output I have some strange indexes
ID=2138 0.1799 0.2761
ID=2137 0.1797 0.2778
ID=2136 0.1796 0.2787
ID=2109 0.1161 0.2072
ID=2007 0.1139 0.2047
The ID is more than 2000 despite I have less than 100 training samples
DBIDs are internal; the documentation clearly says that you shouldn't make too much assumptions on them because their implementation may change. The only reason they are written to the output at all is because some methods (such as OPTICS) may require cross-referencing objects by this unique ID.
Because they are meant to be unique identifiers, they are usually continuously incremented. The next time you click on "run" in the MiniGUI, you will get the next n IDs... so clearly, you clicked run more than once.
The "Tips & Tricks" in the ELKI DBID documentation probably answer your underlying question - how to use map DBIDs to line numbers of your input file. The best way is to if you want to have object identifiers, assign object identifiers yourself by using an identifier column (and configuring it to be an external identifier).
For further information, see the documentation: https://elki-project.github.io/dev/dbids
(Possibly a duplicate of Can't send a keyedMessage to brokers with partitioner.class=kafka.producer.DefaultPartitioner, although the OP of that question didn't mention kafka-python. And anyway, it never got an answer.)
I have a Python program that has been successfully (for many months) sending messages to the Kafka broker, using essentially the following logic:
producer = kafka.KafkaProducer(bootstrap_servers=[some_addr],
retries=3)
...
msg = json.dumps(some_message)
res = producer.send(some_topic, value=msg)
Recently, I tried to upgrade it to send messages to different partitions based on a definite key value extracted from the message:
producer = kafka.KafkaProducer(bootstrap_servers=[some_addr],
key_serializer=str.encode,
retries=3)
...
try:
key = some_message[0]
except:
key = None
msg = json.dumps(some_message)
res = producer.send(some_topic, value=msg, key=key)
However, with this code, no messages ever make it out of the program to the broker. I've verified that the key value extracted from some_message is always a valid string. Presumably I don't need to define my own partitioner, since, according to the documentation:
The default partitioner implementation hashes each non-None key using the same murmur2 algorithm as the java client so that messages with the same key are assigned to the same partition.
Furthermore, with the new code, when I try to determine what happened to my send by calling res.get (to obtain a kafka.FutureRecordMetadata), that call throws a TypeError exception with the message descriptor 'encode' requires a 'str' object but received a 'unicode'.
(As a side question, I'm not exactly sure what I'd do with the FutureRecordMetadata if I were actually able to get it. Based on the kafka-python source code, I assume I'd want to call either its succeeded or its failed method, but the documentation is silent on the point. The documentation does say that the return value of send "resolves to" RecordMetadata, but I haven't been able to figure out, from either the documentation or the code, what "resolves to" means in this context.)
Anyway: I can't be the only person using kafka-python 1.3.3 who's ever tried to send messages with a partitioning key, and I have not seen anything on teh Intertubes describing a similar problem (except for the SO question I referenced at the top of this post).
I'm certainly willing to believe that I'm doing something wrong, but I have no idea what that might be. Is there some additional parameter I need to supply to the KafkaProducer constructor?
The fundamental problem turned out to be that my key value was a unicode, even though I was quite convinced that it was a str. Hence the selection of str.encode for my key_serializer was inappropriate, and was what led to the exception from res.get. Omitting the key_serializer and calling key.encode('utf-8') was enough to get my messages published, and partitioned as expected.
A large contributor to the obscurity of this problem (for me) was that the kafka-python 1.3.3 documentation does not go into any detail on what a FutureRecordMetadata really is, nor what one should expect in the way of exceptions its get method can raise. The sole usage example in the documentation:
# Asynchronous by default
future = producer.send('my-topic', b'raw_bytes')
# Block for 'synchronous' sends
try:
record_metadata = future.get(timeout=10)
except KafkaError:
# Decide what to do if produce request failed...
log.exception()
pass
suggests that the only kind of exception it will raise is KafkaError, which is not true. In fact, get can and will (re-)raise any exception that the asynchronous publishing mechanism encountered in trying to get the message out the door.
I also faced the same error. Once I added json.dumps while sending the key, it worked.
producer.send(topic="first_topic", key=json.dumps(key)
.encode('utf-8'), value=json.dumps(msg)
.encode('utf-8'))
.add_callback(on_send_success).add_errback(on_send_error)
When I use AppleScript to get the properties of an object, a record is returned.
tell application "iPhoto"
properties of album 1
end tell
==> {id:6.442450942E+9, url:"", name:"Events", class:album, type:smart album, parent:missing value, children:{}}
How can I iterate over the key/value pairs of the returned record so that I don't have to know exactly what keys are in the record?
To clarify the question, I need to enumerate the keys and values because I'd like to write a generic AppleScript routine to convert records and lists into JSON which can then be output by the script.
I know it's an old Q but there are possibilities to access the keys and the values now (10.9+). In 10.9 you need to use Scripting libraries to make this run, in 10.10 you can use the code right inside the Script Editor:
use framework "Foundation"
set testRecord to {a:"aaa", b:"bbb", c:"ccc"}
set objCDictionary to current application's NSDictionary's dictionaryWithDictionary:testRecord
set allKeys to objCDictionary's allKeys()
repeat with theKey in allKeys
log theKey as text
log (objCDictionary's valueForKey:theKey) as text
end repeat
This is no hack or workaround. It just uses the "new" ability to access Objective-C-Objects from AppleScript.
Found this Q during searching for other topics and couldn't resist to answer ;-)
Update to deliver JSON functionality:
Of course we can dive deeper into the Foundation classes and use the NSJSONSerialization object:
use framework "Foundation"
set testRecord to {a:"aaa", b:"bbb", c:"ccc"}
set objCDictionary to current application's NSDictionary's dictionaryWithDictionary:testRecord
set {jsonDictionary, anError} to current application's NSJSONSerialization's dataWithJSONObject:objCDictionary options:(current application's NSJSONWritingPrettyPrinted) |error|:(reference)
if jsonDictionary is missing value then
log "An error occured: " & anError as text
else
log (current application's NSString's alloc()'s initWithData:jsonDictionary encoding:(current application's NSUTF8StringEncoding)) as text
end if
Have fun, Michael / Hamburg
If you just want to iterate through the values of the record, you could do something like this:
tell application "iPhoto"
repeat with value in (properties of album 1) as list
log value
end repeat
end tell
But it's not very clear to me what you really want to achieve.
Basically, what AtomicToothbrush and foo said. AppleScript records are more like C structs, with a known list of labels, than like an associative array, with arbitrary keys, and there is no (decent) in-language way to introspect the labels on a record. (And even if there were, you’d still have the problem of applying them to get values.)
In most cases, the answer is “use an associative array library instead.” However, you’re specifically interested in the labels from a properties value, which means we need a hack. The usual one is to force an error using the record, and then parse the error message, something like this:
set x to {a:1, b:2}
try
myRecord as string
on error message e
-- e will be the string “Can’t make {a:1, b:2} into type string”
end
Parsing this, and especially parsing this while allowing for non-English locales, is left as an exercise for the reader.
ShooTerKo's answer is incredibly helpful to me.
I'll bring up another possibility I'm surprised I didn't see anyone else mention, though. I have to go between AppleScript and JSON a lot in my scripts, and if you can install software on the computers that need to run the script, then I highly recommend JSONHelper to basically make the whole problem go away:
https://github.com/isair/JSONHelper