OpenAI Gym suite_gym.load() Loading Error - reinforcement-learning

I am trying to use:
suite_gym.load(env_name)
Registering it with this code:
from gym.envs.registration import register
register(
id='env_name',
entry_point='RL.envs:env',
kwargs={'x': [], 'y': []},
)
gym.make() works perfectly so I know that the environment has successfully registered. However, when I call suite_gym.load() I get this value error:
The gym space None is currently not supported.
In call to configurable 'wrap_env' (<function wrap_env at 0x7fb2f30b3550>)
In call to configurable 'load' (<function load at 0x7fb2f30b35e0>)
I have tried so many different things and I'm really stuck. If you could help with this it would be really appreciated!

I resolved it! So there was an issue with the self.observation_space variable in my environment class! So I fixed that and it no longer registers a None value. If you have this issue later on check to make sure you have a correct self.action_space() and self.observation_space()

Related

Correct way to pass runtime configuration to elixir processes

I'm trying to deploy an app to production and getting a little confused by environment and application variables and what is happening at compile time vs runtime.
In my app, I have a genserver process that requires a token to operate. So I use config/releases.exs to set the token variable at runtime:
# config/releases.exs
import Config
config :my_app, :my_token, System.fetch_env!("MY_TOKEN")
Then I have a bit of code that looks a bit like this:
defmodule MyApp.SomeService do
use SomeBehaviour, token: Application.get_env(:my_app, :my_token),
other_config: :stuff
...
end
In production the genserver process (which does some http stuff) gives me 403 errors suggesting the token isn't there. So can I clarify, is the use keyword getting evaluated at compile time (in which case the application environment doest exist yet)?
If so, what is the correct way of getting runtime environment variables in to a service like this. Is it more correct to define the config in application.ex when starting the process? eg
children = [
{MyApp.SomeService, [
token: Application.get_env(:my_app, :my_token),
other_config: :stuff
]}
...
]
Supervisor.start_link(children, opts)
I may have answered my own questions here, but would be helpful to get someone who knows what they're doing confirm and point me in the right way. Thanks
elixir has two stages: compilation and runtime, both written in Elixir itself. To clearly understand what happens when one should figure out, that everything is macro and Elixir, during compilation stage, expands these macros until everything is expanded. That AST comes to runtime.
In your example, use SomeBehaviour, foo: :bar is implicitly calling SomeBehaviour.__using__/1 macro. To expand the AST, it requires the argument (keyword list) to be expanded as well. Hence, Application.get_env(:my_app, :my_token) call happens in compile time.
There are many possibilities to move it to runtime. If you are the owner of SomeBehaviour, make it accept the pair {:my_app, :my_token} and call Application.get_env/2 somewhere from inside it.
Or, as you suggested, pass it as a parameter to children; this code belongs to function body, meaning it won’t be attempted to expand during compilation stage, but would rather be passed as AST to the resulting BEAM to be executed in runtime.

System.Diagnostic Tracing not working with modules added through autofac.config

I need help with the Autofac.Configuration extension!
We user Autofac and Autofac.Configuration to add all dependencied to our program. Additionally we use the system.diagnositcs Tracing to Log Methode-Calls, etc (the usual).
Now, we only recently started to use the Autofac.Configuration extension. Until then all messages were written to the output file just as expected. However every class that is only added via the configuration file will not be logged at all. (They do work fine in general, just the logging is not working!)
This is what our configuration file looks like:
"components": [
{
"type": "MyClass, MyAssembly",
"services": [
{
"type": "MyInterface, MyInterfaceAssembly"
}
],
"instanceScope": "singleinstance"
},...
This is what our Tracer-Calls look like:
private static readonly TraceSource Tracer = new TraceSource("MyCustomName", SourceLevels.Error);
...
Tracer.TraceInformation($"SomeMessage {someInput}");
Does anyone have an idea what I am doing wrong? Or is this a bug in the Autofac.Configuration extension?
Autofac.Configuration doesn't do anything at all with tracing. If you search through the source there's not even a reference to tracing anywhere in there.
My guess is that something else has happened. Things I'd look at in your shoes:
Error in conversion to configuration: Did I forget to register a class that used to be registered? Did I register all the same parameters and everything that used to be code? Anything at all that's different or that I might have missed?
Unrelated changes: Did I do something else while I was converting to use configuration? Was there other refactoring taking place? Did I pull from the main branch into my task branch and get changes from someone else that may be affecting me?
Unfortunately, you didn't include a minimal reproducible example so there's no way anyone can really help troubleshoot here. However, I can 100% guarantee that the tracing isn't failing due to any bug in the Autofac.Configuration module. It will be something else - either an error in the JSON configuration you've created or some other unrelated change that has occurred.

.NET CORE 3 Upgrade CORS and Json(cycle) XMLHttpRequest Error

I had my working project written in asp.net core 2.1 for a long time, but yesterday, I was forced to upgrade it to .net core 3.0 (due to 2.1 cannot call Dll' s which are written in 3.0 already).
With that, a lot of functions were obsolete or already removed. I fixed almost all of it, but one problem with CORS.
Like many people before me, I used:
app.UseCors(x => x
.AllowAnyOrigin()
.AllowAnyMethod()
.AllowAnyHeader()
.AllowCredentials());
in Configure function. And services.AddCors() in ConfigureServices function.
I was able to fixed this quite easily with setting WithOrigins() or .SetIsOriginAllowed(_ => true) instead of AllowAnyOrigin() which does not work anymore with AllowCredentials().
After that, I was able to start the application and I thought everything is fine, but then I get stuck until now with problem I do not know, how to fix.
I have DB relation N:N and relation table which handle that, that means I have Admin entity with AdminProject list property, then I have AdminProject entity with Admin list and Project list properties and Project entity with AdminProject list property once again.
When I am listing my projects of certain admin, I am returning in Controller this return Ok(projects), where I just use getAll on AdminProject entity and then with Select return only project.
For that, I have to use[JsonIgnore] in project/admin for properties which I do not need to avoid cycling when creating json.
With that said: NOW IN .NET CORE 3.0 AND CORS SETTINGS IT DOES NOT WORK.
I am getting an error:
System.Text.Json.JsonException: A possible object cycle was detected which is not supported. This can either be due to a cycle or if the object depth is larger than the maximum allowed depth of 32.
when debugging in console and error Access to XMLHttpRequest at 'http://localhost:5000/api/project/adminlist/1' from origin 'http://localhost:8080' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. in WEB browser
I think I tried almost everything with Cors settings etc and I do not know why is this happening now. I also tried to JsonConvert.SerializeObject() before return it ---> return Ok(JsonConvert.SerializeObject(projects)) and this is working, but I am not able (mentally) to do this in every single controllers functions.
Please help! Thanks a lot!
The problem was occurring because in .NET Core 3 they change little bit the JSON politics. Json.Net is not longer supported and if you want to used all Json options, you have to download this Nuget: Microsoft.AspNetCore.Mvc.NewtonsoftJson.
After that in your Startup.cs file change/fix/add line where you are adding MVC (in the ConfigureServices method.
So: here is what I did and what fixed my issue:
services.AddMvc(option => option.EnableEndpointRouting = false)
.SetCompatibilityVersion(CompatibilityVersion.Version_3_0)
.AddNewtonsoftJson(opt => opt.SerializerSettings.ReferenceLoopHandling = ReferenceLoopHandling.Ignore);
I hope it will help somebody else.
Cheers!
A couple other things have changed in .net core 3 and now instead of using addMVC you can use addControllers. So your code might look like the follow:
services.AddControllers().AddNewtonsoftJson(x => x.SerializerSettings.ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore);

Uncaught DOMException: Failed to execute 'define' on 'CustomElementRegistry': this name has already been used with this registry

Uncaught DOMException: Failed to execute 'define' on 'CustomElementRegistry': this name has already been used with this registry
at http://127.0.0.1:8000/components/#polymer/polymer/lib/elements/dom-module.js:175:16
Tried deleting node-modules and package-lock and reinstalling did not work.
this error is due to a custom element tag-name being registered which is already registered; to fix simply check that an element by this name hasn't already been registered. This example solution checks to see if something is already registered using the existing API and if not, registers the given Class (inheriting from/extending HTMLElement--at some point):
customElements.get('the-element') || customElements.define('the-element', HTMLTheElement);
For more on the API see https://developer.mozilla.org/docs/Web/API/CustomElementRegistry
most/mature libraries address this problem and those that don't, or are mangled by package and build process complexities can have it pop up; in most cases either updating to a current version, migrating to Lit (https://lit.dev) or patching the problem somehow provides a path to a solution; note the simpler solutions are far easier to maintain--as can be seen in the conflation of npm, polymer over the actual error in the original question; the Polymer project became lit-html and LitElement, and recently rebranded as "Lit" (and still includes these lit-things). Professionally I'm migrating away from npm and Nodejs to Deno with the aim of generally resolve the many problems related to npm and tooling insecurity and complexity, however this answer provides a more direct solution (understand the problem and fix directly, or update to the relevant latest solution which includes this somehow).
Well, this worked for me, with no Typescript warnings,
if (!customElements.get('the-element')) { customElements.define('the-element', HTMLTheElement); }
Hope someone will find this useful.
Cheers.
It is unwise to use the answers above. You want it to fail! The reason being is that your NPM should be deduping duplicate packages, so the fact that you see a certain component being defined on the custom elements registry more than once is a crucial error that you need to debug why the same component is registered more than once.
How to debug, in short, go to your browser, inspect element, network tab, refresh, figure out which files are both registering the same element. Then check in the initiator to see which files are loading those files. Then you get a way better idea of why your app is not resolving the same import to a single place (your deduped dependency).
One reason why you might face this problem is due to semver. If you have multiple different major versions of the same dependency, NPM cannot just dedupe all of the installations to your root node_modules. How you solve this is up to you. Some people use npm-aliases for their different majors of a dependency, some people implement a plugin in their build-tool to resolve paths to a single installation, etc.
For people that can't use #jimmonts answer because the issue is in one of their dependencies you can use the following snippet:
This happens for us, because a package we are using defines an element. But this package is used by multiple apps. And these apps, wouldn't you know it, interact. So customElements.define('x-tag', className) gets called multiple times. And the second time it does, it crashes the app.
function safeDecorator(fn) {
// eslint-disable-next-line func-names
return function(...args) {
try {
return fn.apply(this, args);
} catch (error) {
if (
error instanceof DOMException &&
error.message.includes('has already been used with this registry')
) {
return false;
}
throw error;
}
};
}
customElements.define = safeDecorator(customElements.define);
I was getting the same error. You may not have the same issue as me but I thought I would drop my solution here just incase someone runs into the same issue in the future.
I had two modules that both imported the same custom element module, one of the was importing Module.js and the other module.js. Now the browser saw this as two separate files because URLs can be case sensitive, except my server saw this as one file because it is not case sensitive (express.js) or at least it was able to resolve the path to the correct file even with the incorrect case. And so the browser saw two "different" modules both defining the same custom element, but when I searched my source code only one file was defining the custom element.
I had this problem and found out that I was calling on my boundle.js file twice. Since I was using Webpack and HtmlWebpackPlugin, HtmlWebpackPlugin added the reference to my boundled file to my index.html file where I had already referenced it by hand.
I developed a solution, thats overrite the define with a precheck before define. It works fine for me, just ad the 2 lines into your index.js
customElements.defineclone = Object.assign(Object.create(Object.getPrototypeOf(customElements)).define, customElements);
customElements.define = (name, element) => customElements.get(name) || customElements.defineclone(name, element);

Not able to use built-in models i.e. "user", error: no response from server

I have created many models in loopback for my project. And use these models with the help of "http://localhost:3000/explorer" in my browser.
These model API's are working fine. But, whenever I tried to use built-in models i.e "user" from the explorer. I got the below error:
and in the terminal. I got the below error. And terminal command automatically terminates.
I am new in the loopback. Why am I not able to use built-in models "user".
Please, help me.
Thanks in advance.
To anyone else that encounters this, I ran into this today and found this question unanswered, so I will do my best to give my explanation and findings here.
The error, roleModel.isInRole is not a function, means simply that the role model does not have a function called isInRole.
In my case, this was because another model was automatically generated from an existing database table called Role, which overrode the internal Role model of Loopback.
When the internal Role model is overridden, the isInRole function is not inherited automatically and will result in the above error.