I am running:
select *
from table
and I am getting this error:
Failed with exception java.io.IOException:org.apache.hadoop.hive.serde2.SerDeException: org.codehaus.jackson.JsonParseException: Unexpected character ('O' (code 79)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
at [Source: java.io.StringReader#570e45; line: 1, column: 2]
The tweets are also in a language that I don't know. Is this necessary to have an extension of json in tweets?
You need to change the configuration file in flume. I too faced the issue and resolved by adding the below properties in twitter.conf.
TwitterAgent.sources.Twitter.maxBatchSize = 50000
TwitterAgent.sources.Twitter.maxBatchDurationMillis = 100000
Related
i try to construct JSON with string that contains "\n" in it like this :
ver_str= 'Package ID: version_1234\nBuild\nnumber: 154\nBuilt\n'
proj_ver_str = 'Version_123'
comb = '{"r_content": {0}, "s_version": {1}}'.format(ver_str,proj_ver_str)
json_content = json.loads()
d =json.dumps(json_content )
getting this error:
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:/Dev/python/new_tester/simple_main.py", line 18, in <module>
comb = '{"r_content": {0}, "s_version": {1}}'.format(ver_str,proj_ver_str)
KeyError: '"r_content"'
The error arises not because of newlines in your values, but because of { and } characters in your format string other than the placeholders {0} and {1}. If you want to have an actual { or a } character in your string, double them.
Try replacing the line
comb = '{"r_content": {0}, "s_version": {1}}'.format(ver_str,proj_ver_str)
with
comb = '{{"r_content": {0}, "s_version": {1}}}'.format(ver_str,proj_ver_str)
However, this will give you a different error on the next line, loads() missing 1 required positional argument: 's'. This is because you presumably forgot to pass comb to json.loads().
Replacing json.loads() with json.loads(comb) gives you another error: json.decoder.JSONDecodeError: Expecting value: line 1 column 15 (char 14). This tells you that you've given json.loads malformed JSON to parse. If you print out the value of comb, you see the following:
{"r_content": Package ID: version_1234
Build
number: 154
Built
, "s_version": Version_123}
This isn't valid JSON, because the string values aren't surrounded by quotes. So a JSON parsing error is to be expected.
At this point, let's take a look at what your code is doing and what you seem to want it to do. It seems you want to construct a JSON string from your data, but your code puts together a JSON string from your data, parses it to a dict and then formats it back as a JSON string.
If you want to create a JSON string from your data, it's far simpler to create a dict with your values and use json.dumps on that:
d = json.dumps({"r_content": ver_str, "s_version": proj_ver_str})
I am making use of non-static import
JsonPath jp = response.jsonPath();
System.out.println(jp.get("data?(#.id>14).employee_name").toString());
For a JSON as shown below:
{"status":"success","data":[{"id":"1","employee_name":"Tiger Nixon","employee_salary":"320800","employee_age":"61","profile_image":""},{"id":"2","employee_name":"Garrett Winters","employee_salary":"170750","employee_age":"63","profile_image":""}]}
When i am trying to run it , i am getting below error:
java.lang.IllegalArgumentException: Invalid JSON expression:
Script1.groovy: 1: expecting EOF, found '[' # line 1, column 31.
data[?(#.id>14)].employee_name
^
1 error
Can someone guide me why is this error being thrown ?
I doubt if that syntax is right, nonetheless, you should be using the below
Also note that the id is a string in your response so you will have to include it in quotes
js.get("data.find {it.id > '14'}.employee_name").toString();
I'm trying to design a load test for some APIs, I need to read the data from a CSV file, Here is my CSV file :
amount,description
100,"100 Added"
-150,"-150 removed"
20, "20 added"
the amount is a number.
My CSV data config looks like this:
Image
When I put the body data like this :
Image
I have this error :
{"timestamp":1529427563867,"status":400,"error":"Bad Request","message":"JSON parse error: Unrecognized token 'amount': was expecting ('true', 'false' or 'null'); nested exception is com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'amount': was expecting ('true', 'false' or 'null')\n
and when I change it to this:
Image
I have this error:
"timestamp":1529427739395,"status":400,"error":"Bad Request","message":"JSON parse error: Cannot deserialize value of type `java.math.BigDecimal` from String \"amount\": not a valid representation; nested exception is com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `java.math.BigDecimal` from String \"amount\": not a valid representation\n at [Source: (PushbackInputStream); line: 2, column: 13]
How should I pass the parameters to be able to read from the CSV file?
P.S it works fine when I do all the things without CSV.
In your config, you have "Ignore first line" set to False.
Options:
1: Set "Ignore First Line" to True, for source files with header data included.
2: Leave the config setting as is, but remove the headers from the CSV source file.
While doing Twitter sentiment analysis i come across this error despite searching for extensively for three days for the solution i couldn't get any solution.
Error
hive> select * from load_tweets;
OK
Failed with exception java.io.IOException:org.apache.hadoop.hive.serde2.SerDeException: org.codehaus.jackson.JsonParseException: Unexpected character ('O' (code 79)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
at [Source: java.io.StringReader#5a82bc58; line: 1, column: 2]
Time taken: 1.698 seconds
hive>
Here is the Table creation
hive> create external table load_tweets(id BIGINT,text STRING) ROW FORMAT SERDE 'com.cloudera.hive.serde.JSONSerDe' LOCATION '/user/flume/tweets'
> ;
I am using Cloudera Json serde version
json-serde-1.3.6-SNAPSHOT-jar-with-dependencies.jar
I have checked jar file is properly added in hive class path.
Database:
http://freetexthost.com/ik4jyogkfm
This is database fetch by flume.
I am referring this article for the data fetching
https://acadgild.com/blog/streaming-twitter-data-using-flume/
I am trying to integrate Spring Rest docs with Grails 3.2.9 application, using Rest Assured. During integration testing it is producing the Payload Handling Exception.
Full stack trace:
org.springframework.restdocs.payload.PayloadHandlingException: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
at [Source: [B#793cd42d; line: 1, column: 2]
at org.springframework.restdocs.payload.JsonContentHandler.readContent(JsonContentHandler.java:84)
at org.springframework.restdocs.payload.JsonContentHandler.findMissingFields(JsonContentHandler.java:50)
at org.springframework.restdocs.payload.AbstractFieldsSnippet.validateFieldDocumentation(AbstractFieldsSnippet.java:113)
at org.springframework.restdocs.payload.AbstractFieldsSnippet.createModel(AbstractFieldsSnippet.java:74)
at org.springframework.restdocs.snippet.TemplatedSnippet.document(TemplatedSnippet.java:64)
at org.springframework.restdocs.generate.RestDocumentationGenerator.handle(RestDocumentationGenerator.java:192)
at org.springframework.restdocs.restassured.RestDocumentationFilter.filter(RestDocumentationFilter.java:63)
at com.jayway.restassured.internal.filter.FilterContextImpl.next(FilterContextImpl.groovy:73)
at org.springframework.restdocs.restassured.RestAssuredRestDocumentationConfigurer.filter(RestAssuredRestDocumentationConfigurer.java:65)
at com.jayway.restassured.internal.filter.FilterContextImpl.next(FilterContextImpl.groovy:73)
at com.jayway.restassured.internal.RequestSpecificationImpl.applyPathParamsAndSendRequest(RequestSpecificationImpl.groovy:1574)
at com.jayway.restassured.internal.RequestSpecificationImpl.get(RequestSpecificationImpl.groovy:159)
at so.con.BaseRestDocTestSetup.$tt__createDoc(BaseRestDocTestSetup.groovy:130)
at so.con.BaseRestDocTestSetup.createDoc_closure4(BaseRestDocTestSetup.groovy)
at groovy.lang.Closure.call(Closure.java:414)
at groovy.lang.Closure.call(Closure.java:430)
at grails.transaction.GrailsTransactionTemplate$1.doInTransaction(GrailsTransactionTemplate.groovy:70)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133)
at grails.transaction.GrailsTransactionTemplate.executeAndRollback(GrailsTransactionTemplate.groovy:67)
at so.con.v1.docs.ConfApiDocSpec.test and document get request for /conf getting conf-list as Response(ConfApiDocSpec.groovy:63)
Caused by: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
at [Source: [B#793cd42d; line: 1, column: 2]
at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1702)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:558)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:456)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._handleUnexpectedValue(UTF8StreamJsonParser.java:2689)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._nextTokenNotInObject(UTF8StreamJsonParser.java:878)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:772)
at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:3834)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3783)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2929)
at org.springframework.restdocs.payload.JsonContentHandler.readContent(JsonContentHandler.java:81)
... 19 more
Please help me resolve this.
You are trying to parse XML with JSON decoder. Use Accept-Encoding header for you request and provide expected format there.