Adam optimizer has flaws when used with weight decay. In 2018, AdamW optimizer has been proposed.
Is there any standard way to implement AdamW in MXNet framework (python implementation)? There is mxnet.optimizer.Adam class, but no mxnet.optimizer.AdamW one (checked in mxnet-cu102==1.6.0, mxnet==1.5.0 package versions).
P.S. I asked this questions on MXNet forum and on datascience.stackexchange.com, but to no avail.
Short answer: There isn't a standard way to use AdamW in Gluon yet, but there is some existing work in that direction that would make that relatively easy to add.
Longer answer:
People have been asking for this feature - a lot :) See: https://github.com/apache/incubator-mxnet/issues/9182
Gluon-NLP has a working version of AdamW - possibly slightly different from the one in the original paper: https://github.com/eric-haibin-lin/gluon-nlp/blob/df63e2c2a4d6b998289c25a38ffec8f4ff647ff4/src/gluonnlp/optimizer/bert_adam.py
The adamw_update() operator was added with this pull request: https://github.com/apache/incubator-mxnet/pull/13728 This is first released in MXNet 1.6.0.
Unfortunately, it looks like there isn't a way to use this with gluon.Trainer directly right now, without copying/modifying the BERTAdam code (or writing something similar from scratch). That would be a very nice thing to add to Gluon.
Please let me know if you get this working, as I'd love to be able to use that as well.
For example ,I want to express Vec(0),Vec(1),Vec(2),Vec(3),Vec(4) together.Can I express it Vec(0->4) ?If it's error,what should I do ?
And I also want to know which is the fullest document of the Chisel?I have many problem,but I can't find the answer in my documents.I have Digital Design with Chisel and chisel_cheatsheet.
I want to express Vec(0),Vec(1),Vec(2),Vec(3),Vec(4) together.Can I
express it Vec(0->4) ?If it's error,what should I do ?
I think you are looking for a slicing function for Vec as plenn08 asked.
Chick Markley gave a response for a slice assignement.
I have Digital Design with Chisel and chisel_cheatsheet.
That's a good start. You can also take a look at the chisel-bootcamp which can be done with a simple web browser.
When you're new to Chisel and Scala it can be a little intimidating, but I would suggest taking a look at the API docs and learning how to read them: https://www.chisel-lang.org/api/3.2.0/index.html
In this case, for Vec: https://www.chisel-lang.org/api/3.2.0/chisel3/Vec.html
Vec extends Scala Seq and thus has a massive assortment of useful methods like slice as discussed in the post Fabien linked.
I've created some shapes and connected them with bones, that bones have some names, IKbone1, IKbone2...
so the questions is: How can I access to those bones by name in AS3, because I've tried that in I get an error that there are no object name...
any advice?
A manual
IK structure seems to be rather complex in Flash, you first need to reach the correct armature, then you can get bone by name via IKArmature.getBoneByName(). The main data set in those IK bones is stored in points (joints) instead of actual bones.
My demo application I'm working on has a very long startup routine. The application I'm trying to replace with the new ideas log a lot to the console during that (imagine: "now loading data... reticulating splines... login to third party service...").
After spending the whole day learning DI basically from scratch, I create the whole (!) object graph now with a single call to the container. Thank you, everybody here, btw, for providing so many ideas and amazing answers. This community rocks.
But now, what I want to do is to make initialization deterministic again, so I can log in my workflow (I'm using Workflow Foundation 4.0, because I like the declarative style and the fact that I can show people in graphics what happens) when I load data, reticulate splines and all that.
Do you think it would be an acceptable practice to have a "StartupManager" - class (the only singleton in my architecture now, I killed every other "instance getter"!) that will call secondary initialization methods on the objects it got injected (I used buildUp() and property based DI here)?
Reason is that I want to explicitly call the long initialization methods in my workflow activities. Looks amazing in the editor, my boss will be very happy when I present that (he didn't ask for it, it was my idea to spend the weekend doing something, also I think it is a lot of fun).
i assume you're creating your own DI framework for fun and to learn, right? otherwise just use existing one.
no :) you shouldn't have anything static. your algorithm may look like that:
create instance of your DI builder
feed that instance with dependencies definition (from file or programatically)
call your buildUp on that configured builder. this method should return an instance of context
on the context you call give_me_object_x and you should get an object x filled with all dependencies
or just look how spring is built - it's a very good example of well written DI framework
After reading the nice answers in this question, I watched the screencasts by Justin Etheredge. It all seems very nice, with a minimum of setup you get DI right from your code.
Now the question that creeps up to me is: why would you want to use a DI framework that doesn't use configuration files? Isn't that the whole point of using a DI infrastructure so that you can alter the behaviour (the "strategy", so to speak) after building/releasing/whatever the code?
Can anyone give me a good use case that validates using a non-configured DI like Ninject?
I don't think you want a DI-framework without configuration. I think you want a DI-framework with the configuration you need.
I'll take spring as an example. Back in the "old days" we used to put everything in XML files to make everything configurable.
When switching to fully annotated regime you basically define which component roles yor application contains. So a given
service may for instance have one implementation which is for "regular runtime" where there is another implementation that belongs
in the "Stubbed" version of the application. Furthermore, when wiring for integration tests you may be using a third implementation.
When looking at the problem this way you quickly realize that most applications only contain a very limited set of component roles
in the runtime - these are the things that actually cause different versions of a component to be used. And usually a given implementation of a component is always bound to this role; it is really the reason-of-existence of that implementation.
So if you let the "configuration" simply specify which component roles you require, you can get away without much more configuration at all.
Of course, there's always going to be exceptions, but then you just handle the exceptions instead.
I'm on a path with krosenvold, here, only with less text: Within most applications, you have a exactly one implementation per required "service". We simply don't write applications where each object needs 10 or more implementations of each service. So it would make sense to have a simple way say "this is the default implementation, 99% of all objects using this service will be happy with it".
In tests, you usually use a specific mockup, so no need for any config there either (since you do the wiring manually).
This is what convention-over-configuration is all about. Most of the time, the configuration is simply a dump repeating of something that the DI framework should know already :)
In my apps, I use the class object as the key to look up implementations and the "key" happens to be the default implementation. If my DI framework can't find an override in the config, it will just try to instantiate the key. With over 1000 "services", I need four overrides. That would be a lot of useless XML to write.
With dependency injection unit tests become very simple to set up, because you can inject mocks instead of real objects in your object under test. You don't need configuration for that, just create and injects the mocks in the unit test code.
I received this comment on my blog, from Nate Kohari:
Glad you're considering using Ninject!
Ninject takes the stance that the
configuration of your DI framework is
actually part of your application, and
shouldn't be publicly configurable. If
you want certain bindings to be
configurable, you can easily make your
Ninject modules read your app.config.
Having your bindings in code saves you
from the verbosity of XML, and gives
you type-safety, refactorability, and
intellisense.
you don't even need to use a DI framework to apply the dependency injection pattern. you can simply make use of static factory methods for creating your objects, if you don't need configurability apart from recompiling code.
so it all depends on how configurable you want your application to be. if you want it to be configurable/pluggable without code recompilation, you'll want something you can configure via text or xml files.
I'll second the use of DI for testing. I only really consider using DI at the moment for testing, as our application doesn't require any configuration-based flexibility - it's also far too large to consider at the moment.
DI tends to lead to cleaner, more separated design - and that gives advantages all round.
If you want to change the behavior after a release build, then you will need a DI framework that supports external configurations, yes.
But I can think of other scenarios in which this configuration isn't necessary: for example control the injection of the components in your business logic. Or use a DI framework to make unit testing easier.
You should read about PRISM in .NET (it's best practices to do composite applications in .NET). In these best practices each module "Expose" their implementation type inside a shared container. This way each module has clear responsabilities over "who provide the implementation for this interface". I think it will be clear enough when you will understand how PRISM work.
When you use inversion of control you are helping to make your class do as little as possible. Let's say you have some windows service that waits for files and then performs a series of processes on the file. One of the processes is to convert it to ZIP it then Email it.
public class ZipProcessor : IFileProcessor
{
IZipService ZipService;
IEmailService EmailService;
public void Process(string fileName)
{
ZipService.Zip(fileName, Path.ChangeFileExtension(fileName, ".zip"));
EmailService.SendEmailTo(................);
}
}
Why would this class need to actually do the zipping and the emailing when you could have dedicated classes to do this for you? Obviously you wouldn't, but that's only a lead up to my point :-)
In addition to not implementing the Zip and email why should the class know which class implements the service? If you pass interfaces to the constructor of this processor then it never needs to create an instance of a specific class, it is given everything it needs to do the job.
Using a D.I.C. you can configure which classes implement certain interfaces and then just get it to create an instance for you, it will inject the dependencies into the class.
var processor = Container.Resolve<ZipProcessor>();
So now not only have you cleanly separated the class's functionality from shared functionality, but you have also prevented the consumer/provider from having any explicit knowledge of each other. This makes reading code easier to understand because there are less factors to consider at the same time.
Finally, when unit testing you can pass in mocked dependencies. When you test your ZipProcessor your mocked services will merely assert that the class attempted to send an email rather than it really trying to send one.
//Mock the ZIP
var mockZipService = MockRepository.GenerateMock<IZipService>();
mockZipService.Expect(x => x.Zip("Hello.xml", "Hello.zip"));
//Mock the email send
var mockEmailService = MockRepository.GenerateMock<IEmailService>();
mockEmailService.Expect(x => x.SendEmailTo(.................);
//Test the processor
var testSubject = new ZipProcessor(mockZipService, mockEmailService);
testSubject.Process("Hello.xml");
//Assert it used the services in the correct way
mockZipService.VerifyAlLExpectations();
mockEmailService.VerifyAllExceptions();
So in short. You would want to do it to
01: Prevent consumers from knowing explicitly which provider implements the services it needs, which means there's less to understand at once when you read code.
02: Make unit testing easier.
Pete