lib for configuration handling in an ergonomic way - configuration

I'm writing a common library for loading and handling configuration for my applications,
using the config crate.
Im trying to make it as ergonomic as possible to the user, but can't seem to figure it out.
my lib.rs:
impl RunEnv {
fn to_string(&self) -> String {
match self {
RunEnv::Production => "prod".to_string(),
RunEnv::Dev => "dev".to_string(),
RunEnv::Staging => "stag".to_string(),
}
}
}
impl FromStr for RunEnv {
type Err = String;
fn from_str(s: &str) -> Result<RunEnv, String> {
match s {
"dev" => Ok(RunEnv::Dev),
"stag" => Ok(RunEnv::Staging),
"prod" => Ok(RunEnv::Production),
_ => Err(format!("Could not parse {:?}", s)),
}
}
}
#[derive(Debug, StructOpt)]
#[structopt(name = "CLI Options", about = "Common CLI options for running applications")]
pub struct Arguments {
/// Run in a specific environment mode: dev, stag, prod.
// short and long flags (-e, --env) will be deduced from the field's name
#[structopt(short, long, default_value = "dev")]
pub env: RunEnv,
}
pub trait LoadConfig {
fn new() -> Result<Config, ConfigError>{
const PACKAGE_NAME: &'static str = env!("CARGO_PKG_NAME");
let mut s = Config::new();
let args = Arguments::from_args();
let mut conf_path = String::new();
match args.env {
RunEnv::Production => {
conf_path = format!("/path/to/config/{}/config.toml", PACKAGE_NAME);
}
RunEnv::Staging => {
conf_path = format!("/path/to/config/{}/config.toml", PACKAGE_NAME);
}
RunEnv::Dev => {
conf_path = "tests/config.toml".to_string();
}
}
// Start off by merging in the "default" configuration file
s.merge(File::with_name(&conf_path))?;
// Add in the current environment
// Default to 'dev' env
s.set("run_mode", args.env.to_string())?;
Ok(s)
}
}
In my app:
setting.rs
#[derive(Debug, Deserialize)]
pub struct Server {
port: u32,
address: String,
}
#[derive(Debug, Deserialize)]
pub struct Settings {
server: Server,
run_mode: Option<String>,
}
impl LoadConfig for Settings {}
main.rs
fn main() {
let conf: Settings = Settings::new().unwrap().try_into().unwrap();
println!("{:?}", conf);
And here is my problem, I'm trying to "hide" the.unwrap().try_into().unwrap(); part away
so the lib user will only need to define his setting.rs and run let conf: Settings = Settings::new()
If i move .try_into() inside a trait then I'm getting an error i can't find a way to go around:
| s.try_into()
| ^^^^^^^^ the trait `_IMPL_DESERIALIZE_FOR_Configuration::_serde::Deserialize<'_>` is not implemented for `config::config::Config`
I'm new to rust and probably missing some obvious things

For convention the new function is the way to build an instance and must be an inherent method of the struct,
a method available directly on a type.
In your example you are trying to define new as a trait default method.
If it was at all possible the signature should be:
pub trait LoadConfig {
fn new() -> Self {
}
}
Such trait method is impossible to implement because the trait does not know nothing about Self concrete type.
To follow this convention it is best to rename the LoadConfig::new trait method to something else:
pub trait LoadConfig {
fn load() -> Result<Config, ConfigError>;
}
Then implement the new constructor function as an inherent method, for example:
impl Settings {
fn new() -> Settings {
let config = Settings::load().unwrap(); // TBD: manage errors
let port: u32 = config.get("port").unwrap_or("8080").parse().unwrap();
Settings {
server: Server {
port: port,
address: config.get("address").unwrap_or("localhost").to_owned()
},
run_mode: None
}
}
}
Note that a robust implementation should not unwrap but manage more explicitly config errors.

Related

How to use Rusts tracing_distributed

I am trying to use the Rust tracing_distributed package, but I am getting strange and unhelpful errors when using it, and I am assuming I am using it wrong, but there is no documentation and there are no examples about how to use it. Here is an example of what I'm trying to do :
let trace = tracing_distributed::register_dist_tracing_root(traceId, remote_parent_span_id));
println!("trace value: {:?}", trace);
// the result of trace is: Err(NoEnabledSpan)
I have tried passing a few things in as the traceID and remote_parent_span_id including:
traceId = remote_parent_span_id = Some(tracing::Span::current())
As well as:
traceId = Some(tracing::Span::current())
remote_parent_span_id = ~someParentRequestIdGeneratedUpstream~
I know that the current span is not disabled from trying:
let span = tracing::Span::current();
if span.is_disabled() {
println!("CURRENT SPAN DISABELED");
}
So this leads me to think that the issue is coming from not having the subscriber set properly. I am trying to set the subscriber in an init function which is called before this function which looks like this:
let subscriber = tracing_subscriber::registry() // provide underlying span data store
.with(
tracing_subscriber::fmt::layer()
.json()
.with_span_events(FmtSpan::ACTIVE)
.event_format(stackdriver::StackDriverEventFormat::default())
.with_filter(tracing_subscriber::filter::dynamic_filter_fn(
move |m, c| filter_layer.enabled(m, c.to_owned()),
)),
);
let _ = tracing::subscriber::set_global_default(subscriber)
.map_err(|_err| eprintln!("Unable to set global default subscriber"));
Would anyone be willing to provide me with an example of how to use this library? Or can anyone see what I'm doing wrong here? I have tried everything I can think of.
tracing-distributed has a test which demonstrates how to create and use TelemetryLayer.
I made a demo based on it. In this demo, NoEnabledSpan may be caused by missing #[instrument], which creates a Span for function foo. Hope this will help you find the actual cause.
Also tracing-honeycomb is a great use case for tracing-distributed, better check it out.
use std::sync::{Arc, Mutex};
use tracing::{Id, info};
use tracing::instrument;
use tracing_distributed::{Event, Span, Telemetry, TelemetryLayer};
use tracing_subscriber::layer::SubscriberExt;
use tracing_subscriber::registry;
#[derive(Default, Debug)]
pub struct BlackholeVisitor;
#[derive(PartialEq, Eq, Hash, Copy, Clone, Debug)]
pub struct TraceId(pub(crate) u128);
type SpanId = tracing::Id;
impl tracing::field::Visit for BlackholeVisitor {
fn record_debug(&mut self, _: &tracing::field::Field, _: &dyn std::fmt::Debug) {}
}
/// Mock telemetry capability
pub struct TestTelemetry {
spans: Arc<Mutex<Vec<Span<BlackholeVisitor, SpanId, TraceId>>>>,
events: Arc<Mutex<Vec<Event<BlackholeVisitor, SpanId, TraceId>>>>,
}
impl TestTelemetry {
pub fn new(
spans: Arc<Mutex<Vec<Span<BlackholeVisitor, SpanId, TraceId>>>>,
events: Arc<Mutex<Vec<Event<BlackholeVisitor, SpanId, TraceId>>>>,
) -> Self {
TestTelemetry { spans, events }
}
}
impl Telemetry for TestTelemetry {
type Visitor = BlackholeVisitor;
type TraceId = TraceId;
type SpanId = SpanId;
fn mk_visitor(&self) -> Self::Visitor {
BlackholeVisitor
}
fn report_span(&self, span: Span<BlackholeVisitor, SpanId, TraceId>) {
// succeed or die. failure is unrecoverable (mutex poisoned)
let mut spans = self.spans.lock().unwrap();
spans.push(span);
}
fn report_event(&self, event: Event<BlackholeVisitor, SpanId, TraceId>) {
// succeed or die. failure is unrecoverable (mutex poisoned)
let mut events = self.events.lock().unwrap();
events.push(event);
}
}
#[instrument]
fn foo() {
let trace = tracing_distributed::register_dist_tracing_root(TraceId(123), Option::<Id>::None);
println!("trace value: {:?}", trace);
info!("test");
}
fn main() {
let spans = Arc::new(Mutex::new(Vec::new()));
let events = Arc::new(Mutex::new(Vec::new()));
let cap = TestTelemetry::new(spans.clone(), events.clone());
let telemetry_layer = TelemetryLayer::new("test_svc_name", cap, |x| x);
let subscriber = registry::Registry::default()
.with(tracing_subscriber::fmt::Layer::default())
.with(telemetry_layer);
// dbg!(&subscriber);
tracing::subscriber::set_global_default(subscriber).expect("setting global default failed");
foo();
dbg!(&spans);
dbg!(&events);
}
crate versions:
tracing = "0.1.32"
tracing-distributed = "0.4.0"
tracing-subscriber = "0.3.10"

Is there a way to derive a struct like with Deserialize to get automatic transform from serde_json::Value?

Deserialising from a string directly into a struct works perfectly. But in some cases, you may already have a serde_json::Value in your hands, and want to try and convert it into a struct.
The following example illustrate just that: loading a Request struct from JSON (in a network library for example), with a type string and a generic content as a Value, and then you want to call a handler (from a client library) with the value transformed into a given struct.
use serde::Deserialize;
use serde_json::{json, Value};
use std::convert::TryFrom;
use std::error::Error;
#[derive(Deserialize)]
struct Request {
#[serde(alias = "type")]
req_type: String,
content: Value
}
#[derive(Deserialize)]
struct Person {
name: String,
age: u8
}
// It there a way to avoid having to declare this???
impl TryFrom<Value> for Person {
type Error = serde_json::Error;
fn try_from(value: Value) -> Result<Self, Self::Error> {
Person::deserialize(value)
}
}
fn say_hello(p: Person) {
println!("Hello, I'm {}, and I'm {} years old!", p.name, p.age);
}
fn main() -> Result<(), Box<dyn Error>> {
let req: Request = Request::deserialize(json!({
"type": "sayHello",
"content": {
"name": "Pierre",
"age": 32
}
}))?;
match req.req_type.as_str() {
"sayHello" => say_hello(req.content.try_into()?),
_ => println!("unknown request")
}
Ok(())
}
So the question is: is there some derive or other magic implemented which would allow the same behaviour as Deserialize from String, so that the client can only write:
#[derive(Deserialize)]
struct Person {
name: String,
age: u8
}
fn say_hello(p: Person) {
println!("Hello, I'm {}, and I'm {} years old!", p.name, p.age);
}
I tried the #[serde(try_from = "Value")] attribute but it does not look like it's intended for that purpose...
There is serde_json::from_value() specifically for this:
pub fn from_value<T>(value: Value) -> Result<T, Error>
where
T: DeserializeOwned,
Given any serde_json::Value and some T: DeserializedOwned, the function will deserialize the Value to that T, if possible.

How do I fix "the method `clone` exists but the following trait bounds were not satisfied" when cloning a struct with generics and a function pointer? [duplicate]

This question already has answers here:
Deriving a trait results in unexpected compiler error, but the manual implementation works
(2 answers)
Closed 2 years ago.
I'm writing a function that will create a menu interface in the terminal. To use the function, it needs to be passed a Vec<menu_option>
#[derive(Clone)]
pub struct menu_option<'a, T> {
pub command: &'a str,
pub description: &'a str,
pub action: menu_option_action<'a, T>,
}
#[derive(Clone)]
pub enum menu_option_action<'a, T> {
sub_menu(Vec<menu_option<'a, T>>),
callback(fn(state: T) -> T),
leave,
}
When I design the menu, it can have multiple levels and there can be repeats of a menu_action. Namely, each layer has a "leave"/"return":
// In my actual code I construct a vector of `menu_option`s
fn example() {
type StateType = i32;
let mut state: StateType = 88;
let leave_option: menu_option<&mut StateType> = menu_option::<&mut StateType> {
command: "l",
description: "leave",
action: menu_option_action::leave,
};
let a = leave_option.clone();
}
error[E0599]: no method named `clone` found for struct `menu_option<'_, &mut i32>` in the current scope
--> src/lib.rs:24:26
|
2 | pub struct menu_option<'a, T> {
| -----------------------------
| |
| method `clone` not found for this
| doesn't satisfy `menu_option<'_, &mut i32>: std::clone::Clone`
...
24 | let a = leave_option.clone();
| ^^^^^ method not found in `menu_option<'_, &mut i32>`
|
= note: the method `clone` exists but the following trait bounds were not satisfied:
`&mut i32: std::clone::Clone`
which is required by `menu_option<'_, &mut i32>: std::clone::Clone`
= help: items from traits can only be used if the trait is implemented and in scope
= note: the following trait defines an item `clone`, perhaps you need to implement it:
candidate #1: `std::clone::Clone`
How do I fix this?
I tried searching online and couldn't find a solution, at least, not one that worked for me (what I found was old Rust, I think)
I don't know what stops the clone because the only use of the T type is a function pointer which I thought was cloneable.
It works when you implement Clone yourself on menu_option & menu_option_action. By default, since your struct/enum has a type parameter, the macro expansion of the #[derive(Clone)] attribute will restrain the clone implementation for your type to T: Clone.
In your case, not only is this requirement not needed, but it is also very unlikely to be ever respected (e.g. &mut T does not implement Clone). By implementing Clone by hand, you can get rid of the T: Clone requirement and it works!
Note that function pointers implement Copy which is why the menu_option_action::callback(*f) works:
type StateType = i32;
pub struct menu_option<'a, T> {
pub command: &'a str,
pub description: &'a str,
pub action: menu_option_action<'a, T>,
}
impl<'a, T> Clone for menu_option<'a, T> {
fn clone(&self) -> Self {
menu_option {
command: self.command.clone(),
description: self.description.clone(),
action: self.action.clone(),
}
}
}
pub enum menu_option_action<'a, T> {
sub_menu(Vec<menu_option<'a, T>>),
callback(fn(T) -> T),
leave,
}
impl<'a, T> Clone for menu_option_action<'a, T> {
fn clone(&self) -> Self {
match self {
menu_option_action::sub_menu(sub) => menu_option_action::sub_menu(sub.to_vec()),
menu_option_action::callback(f) => menu_option_action::callback(*f),
menu_option_action::leave => menu_option_action::leave,
}
}
}
fn main() {
let mut state: StateType = 88;
let leave_option: menu_option<&mut StateType> = menu_option {
command: "l",
description: "leave",
action: menu_option_action::leave,
};
let a = leave_option.clone();
}

rustc-serialize custom enum decoding

I have a JSON structure where one of the fields of a struct could be either an object, or that object's ID in the database. Let's say the document looks like this with both possible formats of the struct:
[
{
"name":"pebbles",
"car":1
},
{
"name":"pebbles",
"car":{
"id":1,
"color":"green"
}
}
]
I'm trying to figure out the best way to implement a custom decoder for this. So far, I've tried a few different ways, and I'm currently stuck here:
extern crate rustc_serialize;
use rustc_serialize::{Decodable, Decoder, json};
#[derive(RustcDecodable, Debug)]
struct Car {
id: u64,
color: String
}
#[derive(Debug)]
enum OCar {
Id(u64),
Car(Car)
}
#[derive(Debug)]
struct Person {
name: String,
car: OCar
}
impl Decodable for Person {
fn decode<D: Decoder>(d: &mut D) -> Result<Person, D::Error> {
d.read_struct("root", 2, |d| {
let mut car: OCar;
// What magic must be done here to get the right OCar?
/* I tried something akin to this:
let car = try!(d.read_struct_field("car", 0, |r| {
let r1 = Car::decode(r);
let r2 = u64::decode(r);
// Compare both R1 and R2, but return code for Err() was tricky
}));
*/
/* And this got me furthest */
match d.read_struct_field("car", 0, u64::decode) {
Ok(x) => {
car = OCar::Id(x);
},
Err(_) => {
car = OCar::Car(try!(d.read_struct_field("car", 0, Car::decode)));
}
}
Ok(Person {
name: try!(d.read_struct_field("name", 0, Decodable::decode)),
car: car
})
})
}
}
fn main() {
// Vector of both forms
let input = "[{\"name\":\"pebbles\",\"car\":1},{\"name\":\"pebbles\",\"car\":{\"id\":1,\"color\":\"green\"}}]";
let output: Vec<Person> = json::decode(&input).unwrap();
println!("Debug: {:?}", output);
}
The above panics with an EOL which is a sentinel value rustc-serialize uses on a few of its error enums. Full line is
thread '<main>' panicked at 'called `Result::unwrap()` on an `Err` value: EOF', src/libcore/result.rs:785
What's the right way to do this?
rustc-serialize, or at least its JSON decoder, doesn't support that use case. If you look at the implementation of read_struct_field (or any other method), you can see why: it uses a stack, but when it encounters an error, it doesn't bother to restore the stack to its original state, so when you try to decode the same thing differently, the decoder is operating on an inconsistent stack, eventually leading to an unexpected EOF value.
I would recommend you look into Serde instead. Deserializing in Serde is different: instead of telling the decoder what type you're expecting, and having no clear way to recover if a value is of the wrong type, Serde calls into a visitor that can handle any of the types that Serde supports in the way it wants. This means that Serde will call different methods on the visitor depending on the actual type of the value it parsed. For example, we can handle integers to return an OCar::Id and objects to return an OCar::Car.
Here's a full example:
#![feature(custom_derive, plugin)]
#![plugin(serde_macros)]
extern crate serde;
extern crate serde_json;
use serde::de::{Deserialize, Deserializer, Error, MapVisitor, Visitor};
use serde::de::value::MapVisitorDeserializer;
#[derive(Deserialize, Debug)]
struct Car {
id: u64,
color: String
}
#[derive(Debug)]
enum OCar {
Id(u64),
Car(Car),
}
struct OCarVisitor;
#[derive(Deserialize, Debug)]
struct Person {
name: String,
car: OCar,
}
impl Deserialize for OCar {
fn deserialize<D>(deserializer: &mut D) -> Result<Self, D::Error> where D: Deserializer {
deserializer.deserialize(OCarVisitor)
}
}
impl Visitor for OCarVisitor {
type Value = OCar;
fn visit_u64<E>(&mut self, v: u64) -> Result<Self::Value, E> where E: Error {
Ok(OCar::Id(v))
}
fn visit_map<V>(&mut self, visitor: V) -> Result<Self::Value, V::Error> where V: MapVisitor {
Ok(OCar::Car(try!(Car::deserialize(&mut MapVisitorDeserializer::new(visitor)))))
}
}
fn main() {
// Vector of both forms
let input = "[{\"name\":\"pebbles\",\"car\":1},{\"name\":\"pebbles\",\"car\":{\"id\":1,\"color\":\"green\"}}]";
let output: Vec<Person> = serde_json::from_str(input).unwrap();
println!("Debug: {:?}", output);
}
Output:
Debug: [Person { name: "pebbles", car: Id(1) }, Person { name: "pebbles", car: Car(Car { id: 1, color: "green" }) }]
Cargo.toml:
[dependencies]
serde = "0.7"
serde_json = "0.7"
serde_macros = "0.7"

Omit values that are Option::None when encoding JSON with rustc_serialize

I have a struct that I want to encode to JSON. This struct contains a field with type Option<i32>. Let's say
extern crate rustc_serialize;
use rustc_serialize::json;
#[derive(RustcEncodable)]
struct TestStruct {
test: Option<i32>
}
fn main() {
let object = TestStruct {
test: None
};
let obj = json::encode(&object).unwrap();
println!("{}", obj);
}
This will give me the output
{"test": null}
Is there a convenient way to omit Option fields with value None? In this case I would like to have the output
{}
If someone arrives here with the same question like I did, serde has now an option skip_serializing_none to do exactly that.
https://docs.rs/serde_with/1.8.0/serde_with/attr.skip_serializing_none.html
It doesn't seem to be possible by doing purely from a struct, so i converted the struct into a string, and then converted that into a JSON object. This method requires that all Option types be the same type. I'd recommend if you need to have variable types in the struct to turn them into string's first.
field_vec and field_name_vec have to be filled with all fields at compile time, as I couldn't find a way to get the field values, and field names without knowing them in rust at run time.
extern crate rustc_serialize;
use rustc_serialize::json::Json;
fn main() {
#[derive(RustcEncodable)]
struct TestStruct {
test: Option<i32>
}
impl TestStruct {
fn to_json(&self) -> String {
let mut json_string = String::new();
json_string.push('{');
let field_vec = vec![self.test];
let field_name_vec = vec![stringify!(self.test)];
let mut previous_field = false;
let mut count = 0;
for field in field_vec {
if previous_field {
json_string.push(',');
}
match field {
Some(value) => {
let opt_name = field_name_vec[count].split(". ").collect::<Vec<&str>>()[1];
json_string.push('"');
json_string.push_str(opt_name);
json_string.push('"');
json_string.push(':');
json_string.push_str(&value.to_string());
previous_field = true;
},
None => {},
}
count += 1;
}
json_string.push('}');
json_string
}
}
let object = TestStruct {
test: None
};
let object2 = TestStruct {
test: Some(42)
};
let obj = Json::from_str(&object.to_json()).unwrap();
let obj2 = Json::from_str(&object2.to_json()).unwrap();
println!("{:?}", obj);
println!("{:?}", obj2);
}
To omit Option<T> fields, you can create an implementation of the Encodable trait (instead of using #[derive(RustcEncodable)]).
Here I updated your example to do this.
extern crate rustc_serialize;
use rustc_serialize::json::{ToJson, Json};
use rustc_serialize::{Encodable,json};
use std::collections::BTreeMap;
#[derive(PartialEq, RustcDecodable)]
struct TestStruct {
test: Option<i32>
}
impl Encodable for TestStruct {
fn encode<S: rustc_serialize::Encoder>(&self, s: &mut S) -> Result<(), S::Error> {
self.to_json().encode(s)
}
}
impl ToJson for TestStruct {
fn to_json(&self) -> Json {
let mut d = BTreeMap::new();
match self.test {
Some(value) => { d.insert("test".to_string(), value.to_json()); },
None => {},
}
Json::Object(d)
}
}
fn main() {
let object = TestStruct {
test: None
};
let obj = json::encode(&object).unwrap();
println!("{}", obj);
let decoded: TestStruct = json::decode(&obj).unwrap();
assert!(decoded==object);
}
It would be nice to implement a custom #[derive] macro which does this automatically for Option fields, as this would eliminate the need for such custom implementations of Encodable.