error: the type of this value must be known in this context (Rust) / serde_json Value - json

I am using serde_json to deserialise a json document. I have a function that given a string (this is the json document), will return a serde_json Value (this is an enum that represents the json type), returns an Option.
This value is passed around to other functions as required.
However, I realised that passing around a Value is not quite what I want, because doing this, the key is not available.
To illustrate my point, if I have a json document that looks like this:
{
"root" : {
"regex" : null,
"prefixes" : [ "a_", "b_" ]
}
}
"root" is a json object, "regex" is json Null and "prefixes" is a json array.
Now, the json type Value is an enum with discriminators representing the json types, eg Object, Null, Array for the examples given above.
The serde_json crate uses std::collections::BTreeMap to represent nodes in the json document, where the String type repesents the json keys (in the above, these would be "root", "regex" and "prefixes". So passing around just references to Values is only partly helpful, I should be passing around BTreeMap instead, so that I can access the key too.
So this is the following function that I am trying to re-write:
fn get_json_content(content_s : &str) -> Option<Value> {
// instead of returning a value, we need to return a BTreeMap, so we can get the
// key and the value.
println!("===>>> json_content obtained: {}", content_s);
match serde_json::from_str(content_s) { // -> Result<Value>
Ok(some_value) => Some(some_value),
Err(_) => None
}
}
So I started to re-write the function but became up against the "the type of this value must be known in this context" error:
fn get_json_content_as_btreemap<'a>(content_s : &str) -> Option<&'a BTreeMap<String, Value>> {
match serde_json::from_str(content_s) { // -> Result<Value>
Ok(some) => {
// I expect the type of key_value_pair to be BTreeMap<String, Value>>
// (but I may be wrong!)
let key_value_pair = some.as_object().unwrap(); // Error here
},
Err(_) => None
}
}
I found other questions on stackoverflow like this one:
the type of this value must be known in this context
and using this as a helper, I tried to insert the type as follows:
let key_value_pair = some.as_object::<BTreeMap<_, _>>().unwrap();
which doesnt fix the issue. Also, tried other similar variations to no avail. So how do I fix this please?
EDIT:
I have another function in this app as follows:
fn get_root_value<'a>(json_documemt : &'a Value) -> Result<&'a Value, JsonErrorCode> {
if json_documemt.is_object() {
for (k, v) in json_documemt.as_object().unwrap().iter() {
if k == "root" {
println!("found root: {}", k);
return Ok(v)
}
}
return Err(JsonErrorCode::Custom("Failed to find root node".to_string()))
}
Err(JsonErrorCode::Custom("Not an object".to_string()))
}
... and this works fine. Here you can see that I can call as_object() and then obtain the key and value as a tuple pair. I don't understand why as_object is working in one case but not the other. I would like to pull out the BTreeMap and pass this around as a borrowed item.

You can change the return type of your initial function and serde_json will deserialize to the appropriate object if it can:
fn get_json_content(content_s : &str) -> Option<BTreeMap<String, Value>> {
// instead of returning a value, we need to return a BTreeMap, so we can get the
// key and the value.
println!("===>>> json_content obtained: {}", content_s);
match serde_json::from_str(content_s) { // -> Result<Value>
Ok(some_value) => Some(some_value),
Err(_) => None
}
// Note: this match statement can be rewritten as
// serde_json::from_str(content_s).ok()
}
Your second example won't work because you are instantiating the Value object inside the function, and then trying to return a reference to the object you just instantiated. This won't work because the object will go out of scope at the end of the function and the reference will then be invalid.

Related

how to get value from json via variable in typescript?

supposing I have to read some data from some json files(i18n), every json file may look like:
{
"foo": "1",
"bar": "2",
...
}
I don't know how many fields this json have(it can be expanded), but it's fields look like
{
[prop: string]: string
}
besides, all the json files share the same fields.
when I try to read a value from this json via:
//a can be expanded, I'm not sure how many fileds does it have
let a = {
name: "dd",
addr: "ee",
}
//I'm confident a has a field "name"
let b = "name";
console.log(a[b]);
the error message is:
Element implicitly has an 'any' type because expression of type 'string' can't be used to index type
how could I fix it?
The error you're encountering is because the keys in a is not just any string (in fact, it can only be "name" or "add"), but b can be a string of any arbitrary value. If you are very sure that b represents a key found in the object a, you can hint TypeScript as such:
let b: keyof typeof a = "name";
Attempting to assign any arbitrary string value to b will lead to an error:
// This will cause an error
let b: key typeof a = "foobar";
See proof-of-concept on TypeScript Playground.

What is the fastest correct way to detect that there are no duplicates in a JSON array?

I need to check if all items are unique in an array of serde_json::Value. Since this type does not implement Hash I came up with the following solution:
use serde_json::{json, Value};
use std::collections::HashSet;
fn is_unique(items: &[Value]) -> bool {
let mut seen = HashSet::with_capacity(items.len());
for item in items.iter() {
if !seen.insert(item.to_string()) {
return false;
}
}
true
}
fn main() {
let value1 = json!([1, 2]);
assert!(is_unique(&value1.as_array().unwrap()));
let value2 = json!([1, 1]);
assert!(!is_unique(&value2.as_array().unwrap()));
}
I assume that it should only work if serde_json is built with preserve_order feature (to have objects serialized in the same order every time), but I am not 100% sure about it.
Main usage context:
JSON Schema validation. "uniqueItems" keyword implementation.
Related usage case
Deduplication of JSON arrays to optimize JSON Schema inference on them.
For example, the input data is [1, 2, {"foo": "bar"}]. A straightforward inference might output this:
{
"type": "array",
"items": {
"anyOf": [
{"type": "integer"},
{"type": "integer"},
{"type": "object", "required": ["foo"]}
]
}
}
values in items/anyOf can be reduced to only two values.
Question: What would be the most time-efficient and correct way to check that there are no duplicates in an arbitrary JSON array?
I used serde_json = "1.0.48"
Rust: 1.42.0
Playground
Converting each array item to a string is rather expensive – it requires at least one string allocation per item, and quite likely more than that. It's also difficult to make sure mappings (or "objects" in JSON language) are represented in a canonical form.
A faster and more robust alternative is to implement Hash for Value yourself. You need to define a newtype wrapper, since you can't implement a foreign trait on a foreign type. Here's a simple example implementation:
use serde_json::Value;
use std::hash::{Hash, Hasher};
use std::collections::hash_map::DefaultHasher;
#[derive(PartialEq)]
struct HashValue<'a>(pub &'a Value);
impl Eq for HashValue<'_> {}
impl Hash for HashValue<'_> {
fn hash<H: Hasher>(&self, state: &mut H) {
use Value::*;
match self.0 {
Null => state.write_u32(3_221_225_473), // chosen randomly
Bool(ref b) => b.hash(state),
Number(ref n) => {
if let Some(x) = n.as_u64() {
x.hash(state);
} else if let Some(x) = n.as_i64() {
x.hash(state);
} else if let Some(x) = n.as_f64() {
// `f64` does not implement `Hash`. However, floats in JSON are guaranteed to be
// finite, so we can use the `Hash` implementation in the `ordered-float` crate.
ordered_float::NotNan::new(x).unwrap().hash(state);
}
}
String(ref s) => s.hash(state),
Array(ref v) => {
for x in v {
HashValue(x).hash(state);
}
}
Object(ref map) => {
let mut hash = 0;
for (k, v) in map {
// We have no way of building a new hasher of type `H`, so we
// hardcode using the default hasher of a hash map.
let mut item_hasher = DefaultHasher::new();
k.hash(&mut item_hasher);
HashValue(v).hash(&mut item_hasher);
hash ^= item_hasher.finish();
}
state.write_u64(hash);
}
}
}
}
The value for None is chosen randomly to make it unlikely to collide with other entries. To calculate hashes for floating point numbers, I used the ordered-float crate. For mappings, the code calculates a hash for each key/value pair and simply XORs these hashes together, which is order-independent. It's a bit unfortunate that we need to hardcode the hasher used for hashing the map entries. We could abstract that out by defining our own version of the Hash trait, and then derive concrete implementations of std::hash::Hash from our custom Hash trait, but this complicates the code quite a bit, so I wouldn't do that unless you need to.
We can't derive Eq, since Value does not implement Eq. However, I believe this is just an oversight, so I filed an issue to add an Eq implementation (which the PR has been accepted for, so it will land in some future release).
Depends if JSON array is sorted or not. If it is sorted you can use binary search to check the value is matched with other values. To sort you can use merge sort. Total complexity will be O(nlogn + logn). Or you can iterate sequentially and check for duplicate rows O(n^2).

Why is The Parameter of a Function Retrieved From an Object "never"

I'd expect this code to work. But instead I get this TypeError.
The idea is that myFunctions holds handlers for data received from JSON.
The JSON objects are either of type A, or of type B. If type is "a" I want param to be handled by the function stored in myFunctions.
This is my approach, but the signature of the retrieved function is never allthough all type information is available.
const myFunctions = {
"a": function(o: string) {return "A"},
"b": function(o: number) {return "B"}
};
interface A {
type: "a"
param: string
}
interface B {
type: "b"
param: number
}
function getIt(i: A | B) {
const p = i.param;
const f = myFunctions[i.type];
// at this point typescript identifies the type of f to be ((o: string) => string) | ((o: number) => string)
return f(p); // <- Argument of type 'string | number' is not assignable to parameter of type 'never'. Type 'string' is not assignable to type 'never'.ts(2345)
}
Can someone explain to me why this behaviour occurs and how to fix it?
Alternatively I'd be happy to hear about other approaches to call the correct handler given a certain JSON object.
It is not possible to do this without introducing new if or switch statements. Typescript can't really follow that f and p are related and consistent with one another. Your use case could probably be helped by something like this proposal but that has been sitting as a proposal for a while so I would not really wait for it.
The issue here is that i.type is "A" | "B", so when using it to index myFunctions you just get back a union of all functions (((o: string) => string) | ((o: number) => string)). But this union of functions is only callable with an argument that is an intersection of all possible arguments. That intersection here is string & number which typescript reduces to never since it is a primitive intersection that can never be inhabited by any value. You can read here about the rules on union invocation.
You can add an if or switch to fix this, although it does make the code redundant:
function getIt(i: A | B) {
switch (i.type) {
case "a": return myFunctions[i.type](i.param)
case "b": return myFunctions[i.type](i.param)
}
}
Playground Link
Or use a type assertion to just make things work:
function getIt(i: A | B) {
const p = i.param;
const f = myFunctions[i.type];
return f(p as never);
}
Playground Link

Determining the underlying type of a generic Type with TypeScript

Consider the following interface within TypeScript
interface IApiCall<TResponse> {
method: string;
url: string;
}
Which is then used within the following method;
const call = <TResponse>(api: IApiCall<TResponse>): void => {
// call to API via ajax call
// on response, grab data
// use JSON.parse(data) to convert to json object
return json as TResponse;
};
Now we use this for Type safety within our methods so we know what objects are being returned from the API. However, when we are returning a single string from the API, JSON.parse is converting the string '12345' into a number, which then breaks further down the line when we are trying to treat this as a string and use value.trim() yet it has been translated into a number.
So ideas to solve this so that we are not converting a string into a number.
How can we stop JSON.parse from converting a single string value into a number?
If using JSON.parse, we check the type of TResponse and compare it against the typeof of json generated.
if (typeof (json) !== typeof(TResponse))...
However there doesn't seem to be an obvious way to determine the generic type.
Question 1: How can we stop JSON.parse() from converting a single string value into a number?
JSON is a text format, so in JSON.parse(x), x needs to be a string. But JSON text represents data of not-necessarily-string types. It sounds like you might be making a category mistake, by confusing a thing with its representation.
If you convert the number 12345 to JSON (JSON.stringify(12345)) you will get the string "12345". If you parse that string, (JSON.parse("12345")), you will get the number 12345 back. If you wanted to get the string "12345", you need to encode it as JSON ( JSON.stringify("12345")) as the string "\"12345\"". If you parse that ( JSON.parse('"12345"') you will get the string "12345" out.
So the straightforward answer to the question "How can we stop JSON.parse() from converting a single string value into a number" is "by properly quoting it". But maybe the real problem is that you are using JSON.parse() on something that isn't really JSON at all. If you are given the string "12345" and want to treat it as the string "12345", then you don't want to do anything at all to it... just use it as-is without calling JSON.parse().
Hope that helps. If for some reason either of those don't work for you, you should post more details about your use case as a Minimal, Complete, and Verifiable example.
Question 2: How do we determine that the returned JSON-parsed object matches the generic type?
In TypeScript, the type system exists only at design time and is erased in the emitted JavaScript code that runs later. So you can't access interfaces and type parameters like TResponse at runtime. The general solution to this is to start with the runtime solution (how would you do this in pure JavaScript) and help the compiler infer proper types at design time.
Furthermore, the interface type IApiCall
interface IApiCall<TResponse> {
method: string;
url: string;
}
has no structural dependence on TResponse, which is not recommended. So even if we write good runtime code and try to infer types from it, the compiler will never be able to figure out what TResponse is.
In this case I'd recommend that you make the IApiCall interface include a member which is a type guard function, and then you will have to write your own runtime test for each type you care about. Like this:
interface IApiCall<TResponse> {
method: string;
url: string;
validate: (x: any) => x is TResponse;
}
And here's an example of how to create such a thing for a particular TResponse type:
interface Person {
name: string,
age: number;
}
const personApiCall: IApiCall<Person> = {
method: "GET",
url: "https://example.com/personGrabber",
validate(x): x is Person {
return (typeof x === "object") &&
("name" in x) && (typeof x.name === "string") &&
("age" in x) && (typeof x.age === "number");
}
}
You can see that personApiCall.validate(x) should be a good runtime check for whether or not x matches the Person interface. And then, your call() function can be implemented something like this:
const call = <TResponse>(api: IApiCall<TResponse>): Promise<TResponse | undefined> => {
return fetch(api.url, { method: api.method }).
then(r => r.json()).
then(data => api.validate(data) ? data : undefined);
};
Note that call returns a Promise<Person | undefined> (api calls are probably asynchronous, right? and the undefined is to return something if the validation fails... you can throw an exception instead if you want). Now you can call(personApiCall) and the compiler automatically will understand that the asynchronous result is a Person | undefined:
async function doPersonStuff() {
const person = await call(personApiCall); // no <Person> needed here
if (person) {
// person is known to be of type Person here
console.log(person.name);
console.log(person.age);
} else {
// person is known to be of type undefined here
console.log("File a missing Person report!")
}
}
Okay, I hope those answers give you some direction. Good luck!
Type annotations only exist in TS (TResponse will be nowhere within the output JS), you cannot use them as values. You have to use the type of the actual value, here it should be enough to single out the string, e.g.
if (typeof json == 'string')

Difficulty with pattern matching on an instance of an associated type

I am relying on rustc_serialize to parse JSON data into a struct, Orders, which represents a Vec of Order structs. The JSON data may have an array or a null value; my intent is to either parse the array of orders normally, if any, or parse the null value as an Orders with an empty Vec. If neither of these is the case, then an error is to be relayed. This is my attempt:
impl Decodable for Orders {
fn decode<D: Decoder>(d: &mut D) -> Result<Self, D::Error> {
let try = d.read_seq(|d, l| {
let mut orders = Vec::new();
for _ in 0..l {
let order = try!(Decodable::decode(d));
orders.push(order);
}
Ok(Orders(orders))
});
match try {
value # Ok(_) => value,
error # Err(e) => match e {
ExpectedError(_, x) if &x == "null" => Ok(Orders(Vec::new())),
_ => error,
},
}
}
}
My issue has to do with pattern matching on ExpectedError. The compiler gives me the following error:
expected `<D as rustc_serialize::serialize::Decoder>::Error`,
found `rustc_serialize::json::DecoderError`
(expected associated type,
found enum `rustc_serialize::json::DecoderError`) [E0308]
src/api/types/json.rs:125 ExpectedError(_, x) if &x == "null" => Ok(Orders(Vec::new())),
^~~~~~~~~~~~~~~~~~~
I am stumped on this one. How can I correct this?
How can I correct this?
In general, you would have to choose between being generic or specialized. You cannot pattern match on an associated type because this type can be anything and a generic method should work for any type which satisfies the constraints.
For example, in your case:
<D as rustc_serialize::serialize::Decoder>::Error can be anything
rustc_serialize::json::DecoderError is but one possibility
So you should normally choose between using some abstract D or specializing the decoding for json.
However, since you are implementing Decodable here, you cannot choose NOT to be generic as you do not get to pick the signature.
Furthermore, it appears that rustc_serialize does not support tentative parsing (or at least, its json implementation does not), so I am afraid that you cannot check for nullity (using d.read_nil()) beforehand.
I suppose those limitations can be seen as the reason that this library is being retired in favor of serde, and can only encourage you to try it out.