OK, I have a protobuf formatted data file. I also have a .proto file that describes the schema of the file.
I have found copious libraries that let me extract known messages out of the file. How nice.
However, I don't really know the structure of the file. There may be different top-level "messages" in the file, and what I really want to do is just inspect the file and get a dump of what's in it.
Would love to have a command that lets me do something like:
proto2json <format.proto> <datafile> -o <output.json>
Is this too much to ask for? The Google isn't yielding an obvious answer, so maybe there's something subtle about protobufs I don't get yet.
Ideas?
Thanks to some helpful people on the protocol-buffers google group, I have an answer.
The answer is, "sorry, no".
Well, close. The problem is it's up to you to know what the "root" message is in the data file. In my case it wasn't obvious, so I was hoping a dump of the file would divulge the root. No luck, as the file itself doesn't know what the fields or messages are, they just have data that you can extract if you have the right .proto file.
In my case, I had a few suspicions as to what the root might be, so I did trial and error until I found the message that seemed to know what all of the fields are in the file.
It would have been nice if the .proto file indicated what the root message is, in which case I'm sure a tool to do this conversion would exist already.
I hope this helps.
Here's a pass at solving the problem that you posed. Here's an example command line to run this tool.
$ ./proto2json.sh --schema=test/test.proto \
--root=Recording --in=test/test.pb --out=out.json
https://github.com/rohitsaboo/proto2json
Currently, the tool only supports protocol buffer schemas that do not depend on protos from other files. However, it should be possible to extend it to support "dependency_schemas".
Related
for legal reason I should let the customer be able to download a CSV file but she/he should be able only to read it, not modify it.
What's a common way of handling this use case?
Some kind of signature on the file so that if it's modified you can see it's not in his original form?
I don't need a solution bound to a specific language, I would just like to know what is the best practice.
If customer will be able to download this file into his computer, than you can't stop her/him from modifying it.
However, you may easy detect changes, the easiest will be generating a cryptographic hash function for the file, i.e.:
$ sha256sum data.csv
eea8254c7500ba3de996aa8ad6af399183f04e17d4a8102fde539dbc93a90012 data.csv
I am trying to view the contents of a json file but everything I use it doesn't load.
I tried many sites that claimed to be json viewers, extensions for browser, application and even when trying to edit in notepad++ it seems to be completely unreadble I am not sure if it's obfuscated/encrypted or if I am just doing the wrong thing.
I have tried googling many sources about this and have came to the conclusion that I am not aware of what "type" of json file this is and how to read it. Every application I have used gives data type error, which suggests it is only able to be viewed/compiled in a specific program / method.
I am wondering if anybody can help point me in the right direction! Below I will attach the json file contents
Warning: Large file may take a while to load in browser window
Example of code:
¦¦pÛâTÅbØ*«‹c—¤Î`¯²ÆSú0ÒEX…ÕÊh QDN ‘ùîó/8dzҩݾ 4Ý(úk48–v¹Ôì¯úÓ„é…ƒº¯ÈŸ k"l¾NüžÏ¹úá¾Oð¹ )yà]ŒZš[>øÜáRÜ>¼ksÎÞT,èJ×Àåÿ;+\ L Ù ¯Ki5Uù×]åÁgp
I am sorry if this is not enough information, I am not sure what information is needed from me, I hope I provided enough, if anybody requires viewing the full contents of the json file to help further inspect this I would be happy to :) It is too large to upload to a site like pastebin or others
I have both JSON file and JL file on my computer but when I open them in Notepad their structure looks like the same. What is the difference between them? where shall I use each one?
Actually, the time that I was asking this question I didn't know that "the file type is no guarantee of what is inside it". in other words I thought that for every file name there is a separate manifesto and if a files name is ".something", there is a unique manifesto for it. But now I know that I can create a file, write anything that I want into it and name it ".peyman" and yes there is nothing special with it!
What was that file? the file was JSON Lines file format.
Where did I find it? in the Scrapy except writing scrapy crawl name -o file.json I saw that somebody wrote scrapy crawl name -o file.jl. I tried that and the file was 99% like JSON file so I wondered and asked this question here.
So:
What is the difference between a .JSON file and .JL file? Now I know that the better question is "What is the difference between a .JSON file and .JL file in the Scrapy?"
The JSON Line is like JSON but without the "[" and "]" at the
beginning and the end. it is used in the Scrapy because of this
There's quite a few things that a jl file extension could be referring to. If I remember correctly, it originally had something to do with the window manager Sawfish.
Sawfish was developed in Lisp, and the jl file was a Lisp source file for Sawfish. However, I'm guessing (because you said that inside was JSON-like sauce) that's not what you're asking about.
In that case, I do recall a few projects on GitHub... JSON lambda and Julia.
Both of those may be the reason why you're seeing JSON in a jl file. Without more information on where you got that file, or what it was part of, though, we won't be able to help you much.
That said, file extensions rarely matter in terms of Linux. In Windows, they're far more important, but in Linux you could literally append anything to a file as an "extension" (ie. thisfile.whatever) and you could still open it up in an editor. The same is true for most editors in Windows.
Likely, the packager of that file decided on jl for their own reasons, rather than following convention of using .json.
I guess JL extension is used for many purposes, but JL is also one of the few extensions used for JSON-lines (also known as NDJSON or JSONL).
This format can contain multiple JSON values, one JSON value (with "compact" formatting) per line and is useful for e.g. streaming or logging.
probably a dumb question, but I have a .jsn file that I'm supposed to strip away some unnecessary info from in python and I wanted to make sure .json and .jsn are the same before I proceed. From what I can tell, they are but just wanted to check. Thanks!
JavaScript Object Notation i.e. JSON filenames use the extension .json.
There is a .jsn file format attributed to Shield Now Shield File (JetSoft Corporation) However there is no known link to a real company or file format. In all likely hood it is a simple spelling error. Looking at the format of the file will clearly answer the difference.
I'm about to start writing a program which will attempt to extract data from a Google Code site so that it may be imported in to another project management site. Specifically, I need to extract the full issue detail from the site (description, comments, and so on).
Unfortunately Google don't provide an API for this, nor do they have an export feature, so to me the only option looks to be extracting the data from the actual HTML (yuck). Does any one have any suggestions on "best practice" from attempting to parse data out of HTML? I'm aware that this is less than ideal, but I don't think I have much choice. Can anyone else think of a better way, or maybe someone else has already done this?
Also, I'm aware of the CSV export feature on the issue page, however this does not give complete data about issues (but could be a useful starting point).
I just finished a program called google-code-export (hosted on Github). This allows you to export your Google Code project to an XML file, for example:
>main.py -p synergy-plus -s 1 -c 1
parse: http://code.google.com/p/synergy-plus/issues/detail?id=1
wrote: synergy-plus_google-code-export.xml
... will create a file named synergy-plus_google-code-export.xml.