I am currently testing some SmartDataModels (SmartDestination/Events), and when I try to add an entity in which the description contains some french caracters like "é" or "à", the Orion CB rejects my payload with the following error message :
entityCreate009 400 Bad Request {"error":"BadRequest","description":"Invalid characters in attribute value"}
... because it contains an "é"
The tested JSON payload (NGSI v2) is this one : https://raw.githubusercontent.com/smart-data-models/dataModel.TourismDestinations/2ab2bac0fae5893ba4310714823e293c9030f8c1/Event/examples/example.json
Do I need to provide specific option in order to deal with Locale caracters ?
Many thanks in advance for your help
Rgds
Laurent
Some characters are not allowed in attribute value in order to avoid script injections attack in some circumstances. The set of forbidden characters is described in Orion documentation.
Note "é" or "à" are not in this list, but probably you are using some other that is forbidden and you are getting the error due to this reason.
You can use attribute type TextUnrestricted to avoid this Orion protection mechanism. But use it at your own risk! ;)
Related
The definition for JSON Web Tokens (JWT, see RFC 7519) says that it is a "URL-safe means of representing claims to be transferred between two parties".
I'm wondering, what does it mean if something is URL-safe?
As far as I know, JWT are not passed around as part of the URL. Is it just that, or is there more to it?
Later in the RFC it says:
A JWT is represented as a sequence of URL-safe parts separated by
period ('.') characters. Each part contains a base64url-encoded
value.
This, combined with the RFC not specifying some other meaning explicitly, suggests it means simply "safe to put in a URL" (e.g., doesn't have unencoded / or ? or & characters, etc.).
I have a server written in Rust, this server gets a request in JSON, the JSON the server is getting is a string and sometimes users write quotes inside the value. For example when making a new forum thread.
The only thing I really need to do is to escape the quotes inside the value.
So this:
"{"name":""test"", "username":"tomdrc1", "date_created":"07/12/2019", "category":"Developer", "content":"awdawdasdwd"}"
Needs to be turned into this:
"{"name":"\"test\"", "username":"tomdrc1", "date_created":"07/12/2019", "category":"Developer", "content":"awdawdasdwd"}"
I tried to replace:
let data = let data = "{"name":""test"", "username":"tomdrc1", "date_created":"07/12/2019", "category":"Developer", "content":"awdawdasdwd"}".to_string().replace("\"", "\\\"");
let res: serde_json::Value = serde_json::from_str(&data).unwrap();
But it results in the following error:
thread '' panicked at 'called Result::unwrap() on an Err value: Error("key must be a string", line: 1, column: 2)
I suspect because it transforms the string to the following:
let data = "{\"name\":\"\"test\"\", \"username\":\"tomdrc1\", \"date_created\":\"07/12/2019\", \"category\":\"Developer\", \"content\":\"awdawdasdwd\"}"
If I understand your question right, the issue is that you are receiving strings which should be JSON but are in fact malformed (perhaps generated by concatenating strings).
If you are unable to fix the source of those non-JSON strings the only solution I can think of involves a lot of heavy lifting with caveat:
Writing a custom "malformed-JSON" parser
Careful inspection/testing/analysis of how the broken client is broken
Using the brokenness information to fix the "malformed-JSON"
Using the fixed JSON to do normal request processing
I would recommend not to do that except for maybe a training excercise. Fixing the client will be done in minutes but implementing this perfectly on the server will take days or weeks. The next time this one problematic client has been changed you'll have to redo all the hard work.
The real answer:
Return "400 Bad Request" with some additional "malformed json" hint
Fix the client if you have access to it
Additional notes:
Avoid unwrapping in a server
Look for ways to propagate the Result::Err to caller and use it to trigger a "400 Bad Request" response
Check out error handling chapter in the Rust book for more
I am trying to insert JSON payload into memsql JSON type column but it is failing due to the following reason.
My JSON content having '?' character.
I tried to escape '?' by using the following ways, but it doesn't worked for me.
The Exception i am getting is:
Root Exception stack trace:
java.lang.IndexOutOfBoundsException: Index: 0
Ex payload: "question mark content?"
1. #[org.mule.util.StringUtils.replace(payload,"?","\\?")]
Result: "question mark content\?"
2. #[org.mule.util.StringUtils.replace(payload,"?","\?")]
Result: not allowed to use the above expression
If i use the payload "question mark content" then it is inserted successfully.
Please help me how can I escape '?' in my JSON content while saving it into memsql?
'\?' itself is an escape sequence, so achive this you have to use "\\?" which produce "\?" which should work with memsql.
#[org.mule.util.StringUtils.replace(payload,"?","\\\\?")]
Hope this helps.
From the looks of your exception it looks like you are calling for it to replace the payload, but you're not assigning it to anything.
Going off of the documentation at:
http://grepcode.com/file/repo1.maven.org/maven2/commons-lang/commons-lang/2.4/org/apache/commons/lang/StringUtils.java#3457
It basically says that it's trying to replace the items in a string, and the method itself returns a string. Based on what I can tell in the stack trace, it seems as though you are passing a null or uninitialized variable to something that's trying to parse str[0], which is returning an array out of bounds error.
The way to correct this would be to do something like:
payload = org.mule.util.StringUtils.replace(payload,"?","\\\\?")
Which should replace any instance of ? with \? and re-write it to the payload variable. That said, it sounds like payload may actually be null when you're evaluating it later in your program, which could be indicative of a larger issue.
I’m trying to use the REST API provided by DeployIt (v3.9) to list all the packages available on a given project.
Thus, I use the GET /repository/query service
So, I’m calling this service with the following URL:
http://[server]/deployit/repository/query?namePattern=my-app&type=udm.DeploymentPackage
Unfortunately, I don’t get anything (just an empty list).
If I remove the namePattern from my URL, then I get a long list of all applications (not only the only I'm interested in).
So it appears that I don’t set correctly the namePattern attribute. In the documentation, they say:
a search pattern for the name. This is like the SQL "LIKE" pattern:
the character '%' represents any string of zero or more characters,
and the character '_' (underscore) represents any single character.
Any literal use of these two characters must be escaped with a
backslash ('\'). Consequently, any literal instance of a backslash
must also be escaped, resulting in a double backslash ('\').
So I tried the following URL:
http://[server]/deployit/repository/query?type=udm.DeploymentPackage&namePattern=my-app : empty list
http://[server]/deployit/repository/query?type=udm.DeploymentPackage&namePattern=%my-app%: error 400
http://[server]/deployit/repository/query?type=udm.DeploymentPackage&namePattern=%25my-app%25 (trying to escape the % character): empty list
http://[server]/deployit/repository/query?type=udm.DeploymentPackage&namePattern=Applications/my-app/2.0.0 (with a real version): error, character ‘/’ not allowed.
http://[server]/deployit/repository/query?type=udm.DeploymentPackage&namePattern=2.0.0 : I get the list of all applications deployed with a version 2.0.0 (including my my-app), but that's not what I'm looking for (I want all versions available on DeployIt for my-app).
So, what is the correct URL to retrieve the list of deployed applications?
I've solved my problem. In fact, the namePattern only applies to the last part of the Application name, i.e. the version.
Thus, I have to use the parent attribute to retrieve the list of my application:
http://[server]/deployit/repository/query?type=udm.DeploymentPackage&parent=Applications%2Fmy-app&resultsPerPage=-1
I have a form with method="get". In the form I need to pass the URL of a CSS file but it is encoding it to http%3A%2F%2Fwww... etc.
Is there a way to stop the encoding of the URL as it is breaking the file.
Thanks
Background
It's a bit more subtle than one might think at first sight. For any URL, which is a specific form of the URI standard, certain characters are special. Among the special characters are `:` (scheme separator) and `/` (path or hierarchy separator), here's the full list of reserved symbols from [RFC-2396][1]:
reserved = ";" | "/" | "?" | ":" | "#" | "&" | "=" | "+" |
"$" | ","
It has little to do with security, much more with simply following a standard: these symbols mean something special in any URI, URL or URN. When you need to use them as part of a path or a querystring (the GET request creates a query string for you), you need to escape them. The short version of escaping is: take the UTF-8 bytes as hexadecimal and precede them with a % sign. In the case of the reserved characters, that's always a single-byte character in UTF-8 and thus escaped as two hex digits.
Path to a solution
Back to your problem. You didn't mention what language you were using. But any language that works with the internet has a way of encoding or decoding URLs. Some have helper functions to decode an entire URL, but normally you are better of splitting it into a name/value pairs and then decoding it. This will give you the absolute URL-path you need.
Note: it is best to always decode query values, simply because when people type in a value, they won't know whether that value is reserved, and the browser will encode it for you. Not doing so poses a security risk.
EDIT: When you need to decode within a page, not on the server side, you're going to need JavaScript to do the job. Have a look at this page for en/decoding URLs, or use Google to find many others.
No, you can't. The encoding is required to make a valid URL.
Instead, decode the value in your receiving code (what platform are you on anyways, URL decoding is usually done automatically for you)
If you used XMLHttpRequest you can send text without encoding.
You can use JavaScript to do that, but remember to set content-type to text/plain.
content-type: text/plain
No for security reason you can't do this. You have to collect and decode it at the receiving end.
When you use FORM and GET method and some special chars, you will end up with browser encoding the resulted query.
For newer browsers that support changing the URL address without refreshing the page (IE10+), is possible to decode the URL query string and update the address.
I'm using a script like this:
<script type="text/javascript">
if (history.pushState) { //IE10+
var newurl = window.location.protocol + "//" + window.location.host + window.location.pathname + decodeURIComponent(window.location.search);
window.history.pushState({path:newurl},'',newurl);
}
</script>
This will transform a http://example.com/page.html?path=foo%2Fbar back to http://example.com/page.html?path=foo/bar
You can decode the url using javascript Function: decodeURIComponent(Url );
Because Browser encodes the Url for special characters . For example : https://www.example.com is encoded to %20https%3A%2F%2Fwww.example.com. Here the special characters are replaced by % and its ASCI value.