Say I have an image image.png, and I run some object detection and want to save a json file with annotations as image.png.json. Is there any technical/OS/semantic issue with doing so? Is it considered widely as bad practice? I kind of like the idea of being able to recover the original file extension from the json file.
Related
for legal reason I should let the customer be able to download a CSV file but she/he should be able only to read it, not modify it.
What's a common way of handling this use case?
Some kind of signature on the file so that if it's modified you can see it's not in his original form?
I don't need a solution bound to a specific language, I would just like to know what is the best practice.
If customer will be able to download this file into his computer, than you can't stop her/him from modifying it.
However, you may easy detect changes, the easiest will be generating a cryptographic hash function for the file, i.e.:
$ sha256sum data.csv
eea8254c7500ba3de996aa8ad6af399183f04e17d4a8102fde539dbc93a90012 data.csv
I have both JSON file and JL file on my computer but when I open them in Notepad their structure looks like the same. What is the difference between them? where shall I use each one?
Actually, the time that I was asking this question I didn't know that "the file type is no guarantee of what is inside it". in other words I thought that for every file name there is a separate manifesto and if a files name is ".something", there is a unique manifesto for it. But now I know that I can create a file, write anything that I want into it and name it ".peyman" and yes there is nothing special with it!
What was that file? the file was JSON Lines file format.
Where did I find it? in the Scrapy except writing scrapy crawl name -o file.json I saw that somebody wrote scrapy crawl name -o file.jl. I tried that and the file was 99% like JSON file so I wondered and asked this question here.
So:
What is the difference between a .JSON file and .JL file? Now I know that the better question is "What is the difference between a .JSON file and .JL file in the Scrapy?"
The JSON Line is like JSON but without the "[" and "]" at the
beginning and the end. it is used in the Scrapy because of this
There's quite a few things that a jl file extension could be referring to. If I remember correctly, it originally had something to do with the window manager Sawfish.
Sawfish was developed in Lisp, and the jl file was a Lisp source file for Sawfish. However, I'm guessing (because you said that inside was JSON-like sauce) that's not what you're asking about.
In that case, I do recall a few projects on GitHub... JSON lambda and Julia.
Both of those may be the reason why you're seeing JSON in a jl file. Without more information on where you got that file, or what it was part of, though, we won't be able to help you much.
That said, file extensions rarely matter in terms of Linux. In Windows, they're far more important, but in Linux you could literally append anything to a file as an "extension" (ie. thisfile.whatever) and you could still open it up in an editor. The same is true for most editors in Windows.
Likely, the packager of that file decided on jl for their own reasons, rather than following convention of using .json.
I guess JL extension is used for many purposes, but JL is also one of the few extensions used for JSON-lines (also known as NDJSON or JSONL).
This format can contain multiple JSON values, one JSON value (with "compact" formatting) per line and is useful for e.g. streaming or logging.
I've implemented offline viewing based on the tutorial and github here. The problem I'm having is forge is looking for the db jsons in an odd location that makes url pathing awkward. The files in particular are 'objects_attrs.json.gz', 'objects_vals.json.gz', 'objects_offs.json.gz', 'objects_ids.json.gz', 'objects_avs.json.gz'. For some reason the forge viewer strips two layers of directories off the url then looks for the files there. Afterword Forge looks for the files in the original location but, looks for the straight json instead of the gziped jsons.
This can be handled in a few hacky ways like creating two arbitrary parent directories in the url and accepting files at the higher url as well. Or unzipping the gzips and saving them in location. But, these kind of hacks could easily be unstable if I'm not exactly right about the forge implementation.
Is there any reason these files are searched for in a different location? For example are these files also used by the 2d files not just the 3d files? Are there other files like this?
Look for the .svf file, say it's Design.svf. Rename it to Design.zip, and then upzip the file. This will get you the manifest.json.
Inside the manifest.json file, you'll file the URI's of all the files, including the one's you mentioned (the property database files).
Modify the URI as you wish, save the manifest.json file and re-zip it back up and rename it back to Design.svf.
If you got the URI path matching your file-system, then those property files will now load correctly.
Good luck!
I'm working on an ETL job that will ingest JSON files into a RDS staging table. The crawler I've configured classifies JSON files without issue as long as they are under 1MB in size. If I minify a file (instead of pretty print) it will classify the file without issue if the result is under 1MB.
I'm having trouble coming up with a workaround. I tried converting the JSON to BSON or GZIPing the JSON file but it is still classified as UNKNOWN.
Has anyone else run into this issue? Is there a better way to do this?
I have two json files which are 42mb and 16mb, partitioned on S3 as path:
s3://bucket/stg/year/month/_0.json
s3://bucket/stg/year/month/_1.json
I had the same problem as you, crawler classification as UNKNOWN.
I were able to solved it:
You must create custom classifier with jsonPath as "$[*]" then create new crawler with the classifier.
Run your new crawler with the data on S3 and proper schema will be created.
DO NOT update your current crawler with the classifier as it won't apply the change, I don't know why, maybe because of classifier versioning AWS mentioned in their documents. Create new crawler make them work
As mentioned in
https://docs.aws.amazon.com/glue/latest/dg/custom-classifier.html#custom-classifier-json
When you run a crawler using the built-in JSON classifier, the entire file is used to define the schema. Because you don’t specify a JSON path, the crawler treats the data as one object, that is, just an array.
That is something which Dung also pointed out in his answer.
Please also note that file encoding can lead to JSON being classified as UNKNOWN. Please try and re-encode the file as UTF-8.
probably a dumb question, but I have a .jsn file that I'm supposed to strip away some unnecessary info from in python and I wanted to make sure .json and .jsn are the same before I proceed. From what I can tell, they are but just wanted to check. Thanks!
JavaScript Object Notation i.e. JSON filenames use the extension .json.
There is a .jsn file format attributed to Shield Now Shield File (JetSoft Corporation) However there is no known link to a real company or file format. In all likely hood it is a simple spelling error. Looking at the format of the file will clearly answer the difference.