I'm working on reading data stored in text and csv files using python and process them to create a new JSON document and all works fine except for the content type is application/octet-stream but when I create the JSON document I want this file content type to be application/json.
Data in json_document variable
[
{
"URL":"http://myexample.net",
"domain":"abc",
"domainSerialId":"skdksj12391ncjsacn",
"proxy":"myexample.net"
},
{
"URL":"http://myexample.org",
"domain":"def",
"domainSerialId":"sakdjsaye132978ejwdnwd",
"proxy":"myexample.org"
}
]
json.dumps(json_document)
This is the new consolidated JSON document that gets created. Having issue trying to convert the content type using json.dumps. I upload these documents to Azure blob storage and see the file Content Type to be different.
Assuming you are using the Blob Service API, you need to specify the content_type in your request, otherwise, the default is application/octet-stream.
https://learn.microsoft.com/en-us/rest/api/storageservices/put-blob
Here is a code example using the Azure Python SDK:
# Instantiate a BlobServiceClient using a connection string
from azure.storage.blob import BlobServiceClient
blob_service_client = BlobServiceClient.from_connection_string(self.connection_string)
# Instantiate a ContainerClient
container_client = blob_service_client.get_container_client("mynewcontainer")
# Set mime-type and upload file
content_settings = ContentSettings(content_type='application/json')
container_client.upload_blob(name=dest_path, data=contents.encode('utf-8'),overwrite=True, content_settings=content_settings)
Related
I am using jersey implementation for creating a REST service and trying to upload .json file to the server using form parameters.
I am creating a FileDataBodyPart object and passing it to object of FormDataMultiPart.
FileDataBodyPart fileBodyPart = new FileDataBodyPart(name, file);
form.bodyPart(fileBodyPart);
The default MediaType that is getting set here is application/octet-stream. I looked at jersey library code, it internally "decides" which mediatype to set. If the file type is not in the list of CommonMediaTypes then it sets it as octect-stream.
https://github.com/eclipse-ee4j/jersey/blob/21b7cc0307d6432eb301f773d5c2c674425c6b10/media/multipart/src/main/java/org/glassfish/jersey/media/multipart/file/DefaultMediaTypePredictor.java#L136
The above URL points to the jersey code which lists available mediatypes.
application/json is not in the list of supported formats.
I wanted to know if there is a specific reason for not supporting the json format by jersey library? Or
The support for application/json is added in more recent versions of jersey for multipart file upload.
I am uploading a static site using the databricks platform specifically using the below command for pushing html content to a location.
dbutils.fs.put("/mnt/$web/index.html", html, overwrite=True)
This is working but the HTML file is downloading instead of displaying. This is because the content type is wrong: Content-Type: application/octet-stream.
Is there any way to set this using databricks ?
dbutils.fs.put works with files on DBFS and doesn't "know" about underlying implementation details, because you can mount different things - S3, ADLSv1/v2, etc. Changing of the content-type is specific to the blob storage API, so you will need to implement the code in Python (for example) or Scala that will use that API to set content-type for uploaded files, or upload files & set content-type via API, without dbutils.fs.put.
Finally, this code worked for me. First, I am getting connection string from databricks scope as
dbutils.secrets.get(scope = "generic-scope", key = "website-key")
If you don't have it then look for it inside Storage Account's Container Access Key
from azure.storage.blob import BlobServiceClient, ContentSettings
connect_str="connectionString"
blob_service_client = BlobServiceClient.from_connection_string(connect_str)
# Instantiate a ContainerClient
container_client = blob_service_client.get_container_client("$web")
# List files in blob folder
blobs_list = container_client.list_blobs()
for blob in blobs_list:
print(blob.content_settings.content_type) # application/octet-stream
blob.set_http_headers(
content_settings=ContentSettings(
content_type="text/html; charset=utf-8"
)
)
I am new to WSO2. I wrote Custom Java Class Mediator to transform JSON request to CSV format. I have a proxy service to SFTP the generated CSV file to a (MoveIT) folder.
Custom Mediator converts the JSON request properly to CSV format. But, when send the CSV file using transport.vfs.replyfilename to the endpoint, I see the 'text' wrapper as below in the CSV file:
{"text":"1,F20175_A.CSV,20200623135039\n2,123456789,2017-MO-BX-0048,123456789,987654321,Y/N,C/A,20190101,20201231,20190930,75000,44475.86,15563.52,0.00,15563.52,0.00,60039.38,14960.62,60039.38,0.00,20191218\n3,1,999999999\n"}
I set contentType, MessageType properties "text/plain". I also used vfs.ContentType to set to text/plain as below:
text/plain
I know I am missing something. Has anybody come across this issue in WSO EI 6.6? Should I go ahead and write Custom Message Formatter? Any tips?
I want the output to be written as:
1,F20169_A.CSV,20200617153638
2,123456789,2017-MO-BX-0048,123456789,987654321,Y/N,C/A,20190101,20201231,20190930,75000,44475.86,15563.52,0.00,15563.52,0.00,60039.38,14960.62,60039.38,0.00,20191218
3,1,999999999
Thanks!
It seems that after converting to the CSV format the result is available within a text tag. Therefore using the file connector [1] when you write to the file access the text content using //text() XPath. Please refer to the following sample configuration of inputContent.
<fileconnector.create>
<filePath>{$ctx:filePath}</filePath>
<inputContent>{//text()}</inputContent>
<encoding>{$ctx:encoding}</encoding>
<setTimeout>{$ctx:setTimeout}</setTimeout>
<setPassiveMode>{$ctx:setPassiveMode}</setPassiveMode>
<setSoTimeout>{$ctx:setSoTimeout}</setSoTimeout>
<setUserDirIsRoot>{$ctx:setUserDirIsRoot}</setUserDirIsRoot>
<setStrictHostKeyChecking>{$ctx:setStrictHostKeyChecking}</setStrictHostKeyChecking>
</fileconnector.create>
[1]-https://docs.wso2.com/display/ESBCONNECTORS/Working+with+the+File+Connector#WorkingwiththeFileConnector-create
I have recorded my script to upload json file through Jmeter, but now I am facing the problem while uploading json file through jmeter which is on my local drive.
I am already done with following steps:
Either use full path to file, you're uploading, or copy it to JMeter's "bin" folder
Don't forget to tick Use multipart/form-data for HTTP POST box
Make sure you provide correct Parameter name and MIME Type
Getting exception in my response data:
{"Exception":"Object reference not set to an instance of an object."}
For me works like this. JSON example May be Your server prefer some specific headers? E.g. content type and so on
Why you're using json file?
If you facing problems with generating random JSON data, here is a JSR223 preprocessor:
import org.apache.commons.io.FileUtils;
def time = vars.get("stratup_time")
File myFile = File.createTempFile("upload-", "" );
// Generate Random length string and write to file
FileUtils.write(myFile, "{\"test\":\"$time\"", "UTF-8" )
// Store file name in variable.
vars.put( "filename", myFile.getCanonicalPath() )
And PostProcessor:
import org.apache.commons.io.FileUtils;
// Delete file and do not throw error
FileUtils.deleteQuietly(new File( vars.get("filename")));
In order to be able to record the file upload event you need to copy your .json file to JMeter's "bin" folder, this way HTTP(S) Test Script Recorder will be able to capture the request and generate necessary HTTP Request sampler. It will also populate the HTTP Header Manager
More information: Recording File Uploads with JMeter
I am trying to make a post rest call to my service. My sample input json file is,
{
"$id": "1",
"description": "sfdasd"
}
I have one csv file which contain a bunch of id and description, So is there a option where I can convert csv file to json objects and pass them to post call?
Assuming your CSV file is called test.csv, located in JMeter's "bin" folder and looks like:
Add CSV Data Set Config to your Test Plan and configure it as follows:
You can inline the defined JMeter Variables directly into your request body like:
{
"$id": "${id}",
"description": "${description}"
}
So when you run the test the variables placeholders will automatically be substituted with the values from the CSV file in the HTTP Request sampler:
See Using CSV DATA SET CONFIG article for more information on JMeter tests parameterization using CSV files.
Json is just text. Send as is with the variable id taken from csv:
{ "${id}": "1", "description": "sfdasd" }
CSV data can be converted to JSON via a POJO using Jackson. If a POJO is not already defined or required, you can always use the Java Collection classes to store parsed data and later convert it to JSON. http://www.novixys.com/blog/convert-csv-json-java/ is a good reference on how to convert csv to java.
You can check the following links too.
1. directly convert CSV file to JSON file using the Jackson library
2. Converting an CSV file to a JSON object in Java