Reinforcement Learning textbook or online resources - reinforcement-learning

Dear respected members,
I would like to cover RL in my data analystics course and I am searching for the best resource for students. Do you have any recommendations on textbooks or online resources that benefit students in learning RL?
Appreciated.

My favorite resources were
Sutton & Barto's book
The 1st edition was the standard textbook, and the 2nd edition is recently updated (2020). Sutton hosts the complete PDF on his website.
David Silver has full video lectures available on YouTube. Here is the first one and this playlist has them in order.
One caveat: it's been I while since I studied the topic, I wouldn't know about recent resources.

Related

collecting infos about SAP ERP

I want to start courses on SAP ERP but I have no idea what is it about or where should I start.
Of course I did search online and got an overview but the number of courses in SAP website are just a lot!
I would be very happy if someone can tell me, based on my educational background, which course shroud I take to guarantee my future job.
Thank you
Guru99’s ‘What is SAP?’ guide will give you a good introduction to SAP, as well as detailing the steps to getting certified, and getting employed as an SAP consultant. This makes it a great introduction to the field, with both theoretical background and practical steps.
In terms of advising on specific courses, this would very much depend on you deciding the specific area of SAP you'd like to go into. As this is something you’re new to, I hope the following helps provide you with the context you need to make a decision about which area of SAP work might be right for you.
Assuming you do not have access to the SAP Learning Hub, one of the best resources to learn about SAP is OpenSAP. It has a huge back catalogue covering all technical and functional areas of SAP. Anybody can sign up to any of their courses and certificates as evidence of your learning are available. If the course is currently being run, you can get a certificate for free, otherwise you’ll have to pay a small fee for a certificate. They recently ran a course called ‘Two-Tier ERP with SAP S/4HANA Cloud’, looking at explaining the entire system for SAP ERP, which might be of interest to you. This course should provide an overview, as well as familiarising you with the most up to date technologies used.
If you're looking to work on SAP ERP on a technical level, an essential skill will be ABAP programming. The 'SAP ABAP Programming For Beginners' course on Udemy provides a comprehensive introduction to the language, and how it is used in SAP.
In addition to looking at courses, I’d recommend taking advantage of the wealth of resources available on SAP’s websites. SAP provides roadmaps for its products, which will give you a sense of how to practically apply the knowledge you learn on your courses. I’d also recommend looking through the SAP Community Network, which, like Stack Overflow, has a wiki, a Q&A platform, and a wealth of blog posts detailing various SAP concepts.

Development, hosting and administration HTML5 games and shopping system

I am working for a start-up company that is building a platform of HTML5 gambling games. I was asked to suggest the server technology and hosting solution that should be used. The games need to be able to communicate, allowing among other things for a system of credits that will allow shopping within the site. What do you recommend? Please list all technologies/frameworks/etc you recommend as an integrated solution for such a platform of games - playable on any device.
Also please recommend books that could be helpful with development, hosting and administration of it all.
Lets breakdown your question into simpler parts. Many technologies can help in each of these parts. I will list the most popular open source ones.
Ruby on Rails: For fast development and deploying of scalable web apps.
Database: Many options available depending on RDBMS approach or NoSQL approach. Production level performance examples include sqlite, mongoDB etc.
Twitter Bootstrap: For the best front-end GUI experience on multiple devices.
CMS : Joomla, Drupal for getting a quick start. Not recommended for very fast scalability.
Infrastructure: No one beats Amazon in prices! You have many solutions to choose from and also pay only for the infrastructure that you use.
In addition, follow the development blogs of hugely successful gaming corporations (Rovio, Zynga etc.) to get more details of the stacks they use.
If you can tell me details of technologies that are already used in your start-up, I can zoom in on solutions which will be faster for your developers to switch to.

Recommendation for where to learn web development in a classroom setting

I've been doing server-side development for several years, and have not had much (read - practically no) web development experience. My current employer is flush with ca$h at the moment and is offering to pay for training, so I thought it would be a great opportunity to learn. I seem to learn best in a classroom setting with practice at home, so I was wondering if anyone had suggestions as to where would be a good place to learn this stuff. There are some national firms that offer all kinds of training courses (e.g., LearningTree), but I have no idea how good they are. Local college classes are an option too, as long as I don't have to go through a whole degree curriculum (I'm based in NYC/NJ).
Stuff I'd like to learn includes Servlets/JSPs (starting at a very basic level but quickly moving through advanced), Tomcat, MVC, and integration with frameworks such as Spring. I realize not all of that may come bundled together in one neat little package, but if got 70% of the way there that would be a win too.
Thanks in advance for any and all suggestions.
I have friends who teach for Learning Tree, and they generally seem to know their stuff, but they spend most of their time teaching and working on small projects instead of large/enterprise application development. Since they need at least one week a month of time to teach a course... they can't always get hands-on experience in the bigger projects.
That said, if my company were flush on cash:
I'd try a single course at the most-convenient location near me. Probably Learning Tree, as I respect the two instructors I know who work there.
But mainly, I'd simply ask for time off my primary projects to learn the technologies on my own. If you install Tomcat and get a Hello World servlet built, then try to build an online cookbook site using Spring MVC on Tomcat, you'll have a really good knowledge of things in 40-80 hours.
You'll also have built the skills to continue learning on your own without paying someone else $3k a week to walk you through tutorials you can already find online.

What are greenfield and brownfield applications?

I read the following sentence in the Fluent NHibernate wiki:
...; however, for most greenfield applications (and quite a few brownfield ones too) auto mapping will be more than capable.
What are greenfield and brownfield applications?
Greenfield
in other disciplines like software engineering, a greenfield is also a project which lacks any constraints imposed by prior work. The analogy is to that of construction on greenfield land where there is no need to remodel or demolish an existing structure.
(from http://en.wikipedia.org/wiki/Greenfield_project)
Brownfield
Brownfield development is a term commonly used in the IT industry to describe problem spaces needing the development and deployment of new software systems in the immediate presence of existing (legacy) software applications/systems. This implies that any new software architecture must take into account and coexist with live software already in situ.
(from http://en.wikipedia.org/wiki/Brownfield_(software_development))
I think it might be related to the urban planning terms "greenfield land" and "brownfield land". Greenfield land is undeveloped land - agricultural, landscaping, or natural. Brownfield land is developed land.
A brownfield application is an existing application, while a greenfield application is one that is not yet made or is in very early stages of development.
Greenfield apps: new development, no prior work done that poses constraints on your solution. I think the term comes from un "unplowed, green" field.
Brownfield: existing application, lots of existing stuff to consider, etc.
See this post.
I would guess it's an analogy to building
a greenfield site is virgin ground - i.e. a new project, starting a new software project from scratch
a brownfield site is one where existing structures need to dismantled first, i.e. building within an existing software project
Based on what I know of James Gregory (the guy behind Fluent NHibernate), I think the Wikipedia entry for brownfield is a little on the theoretical side. In Brownfield Application Development, we define it as:
a project, or codebase, that was
previously created and may be
contaminated by poor practices,
structure, and design but has the
potential to be revived through
comprehensive and directed refactoring
Short version: An existing project that needs work but is still actively developed (unlike most legacy systems).
There is much debate surrounding a company's decision to 'greenfield' or 'brownfield' or refactor legacy code.
The decision needs to be made in consideration of many factors - not the least being the available funds to develop a greenfield application. In many cases the legacy app is the company's current cash cow and any greenfield replacement of that legacy code won't make a single red dollar until it is has been fully developed and in the hands of the first paying customer.
While the preference for most software engineers is START NEW GREENFIELD PROJECT because they usually hate OPC (Other People's Code) it may not always be in the long term financial of the company.
I wrote an article that analyses the risks involved with greenfield projects basing it on a very real experience at a company I worked for 20+ years ago (showing my age now ;). You can read it here:
http://stepaheadsoftware.blogspot.com.au/2012/09/greenfield-or-refactor-legacy-code-base.html

Is there a decent open-source gaming console [closed]

Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I've got a young nephew who aspires to grow up to be a game programmer and i'd like to introduce him to the world of open-source as well as get him a sweet gift.
Anything like that out there?
Well, this is a tricky question because we don't know the level your nephew is at, nevermind the fact that it's difficult to produce a very nice showy game without a lot more work than a beginner might put forth.
X Game Station
Nevertheless, André LaMothe's X Game Station is meant to be exactly the system you're asking for - a beginner's guide and system on how to develop complex programs with interactive elements and gameplay on resource limited hardware. Which is pretty much what a game designer is called on to do.
GP32
The GP32 was also meant to fill this gap, but with a much more powerful processor. The successor was never released, and the company went bankrupt shortly after, but you may still be able to find one on ebay or within the communities that developed around the original machine.
Google Android
You might also consider looking toward the Google Android platform. Cell phone gaming is now and will be one of the biggest platforms in the future. The android isn't set up perfectly for gaming, but it's a good first approximation, isn't horribly expensive and includes a robust development toolset for a high-end mobile processor. Several big name game development companies have already pledged support for this platform, so it will also look good on a resume.
But a cheap computer and a VGA graphics book is surprisingly fun as a kid...
-Adam
There are a number of Open Source platforms out their for game development, if you go to here there are a large number of game engines and development platforms. For a beginner with little programming experience I would suggest a game engine like Game Maker or RPG Maker, which are drag and drop game engines. Both of these are free open source game engines. The other problem with these two game engines is that they are limited to 2D games only, even though Game Maker allows you to make and FPS that is like Doom style graphics.
For a more intermediate or beginning programmer I would honestly suggest Torque, which has both a 2D and 3D game engine. This engine is licensed as open source, but you still have to pay for the compiled version from their site. I have had much success with Torque in the past, especially Torque Game Builder (2D Game Engine). It is very simple to pickup and begin making good looking/functional games. In fact, a number of XBOX live games have been built using Torque game engines, like Marble Blast.
Another open source game engine I have heard good things about, but have not had a chance to try, is Multiverse. Multiverse is actually aimed at MMOG creation. The nice thing about Multiverse is that it provides for the creation of a 3D game environment, but also allows for the integration of Flash content into the game world.
For a more advanced programmer I would suggest looking into the SDL, OpenGL, and OpenAL. These are not game engines but graphics and sound libraries for game programmers. These are completely open source and are free to use. Most game technologies will have some implementation of one or all of these libraries in their software.
Even though XNA is not open source, it is also a good place to start for more advanced programmers. Not only does it allow for the creation of XBOX games, but you can also develop for the PC and Zune also.
Another thing you may want to suggest to your nephew is to modify existing games he owns. Most PC games and many console games allow for game modification of some form or another (level design, rescripting, etc). Some of the more popular game engines that I have seen mods for are the Source Engine and Unreal Engine. There are a number of tutorials at MODDB and 3DBuzz (which also has great tutorials on other aspects of game programming and design).
In addition to what I have listed, I have also heard good things about Ogre3D and Havok (a physics engine used in many many games). He can also go to such sites as IGDA, Gamasutra, GameDev,Game Career Guie, 3D Buzz for additional information on game development.
Hope this information helps.
I saw a question earlier about programming on a Nintendo DS. That sounds like it might be what you're looking for.
Also, I recently read about the BUG which looks like a really cool platform for building any number of handheld devices.
You might be interested in the XGameStation. It's a hardware console designed to teach programming a game console. It was created by Andre LaMothe who has written several books on game programming.
http://devmaster.net/devdb/engines/sylphis-3d#general-overview looks pretty nice, c/c++ oriented, GPL license, and Free.
I'd third the Nintendo DS recommendation - grab a R4 "homebrew" cartridge and you're pretty much set.
Another idea is one of the independent handhelds - something like the GP32, though there's certainly newer devices on the market.
I believe this is somewhat like Basketball, start from close to the basket and work your way out and you'll be hitting 3's with practice.
In my opinion, game programming is 3pt, without learning to shoot the basketball properly, you will probably cheat and start slinging it or just chucking the ball at the backboard hoping it's going to go in.
If you have a youngster without the ability to shoot a mid-range basket properly, do you think he will be able to motivate himself to keep trying, rather then trying something closer to the goal and working himself outside when he is confident?
If your nephew is serious about learning programming, get him a python book or vb.net/c# book. Maybe he will become more interested in application development because of these languages (it was the case with me, I'm 14). :)
Edit: This is assuming he doesn't have much programming experience.
well, this is not completely open source (the editor isn't, the engine is), but I recommend RPG Maker VX (if he likes role playing games):
http://tkool.jp/products/rpgvx/eng/index.html
I have used this (and the earlier versions) for some time. It is nice because there is a great editor and event system which can teach basic programming concepts without writing any code. Once your nephew wants to write some real code, there is an entire API (RGSS2) written in ruby that they he is free to manipulate and extend. This API makes up most of the game engine, hiding only low level implementation stuff.
There are other programs out there like GameMaker, but RPG Maker is the most user friendly, while still providing a way to get at the more advanced stuff.
EDIT: I forgot to mention one of the best parts; there is a large and active community around RPG Maker. There are many forums completely dedicated to the program where people will be happy to help on even the most basic tasks, not to mention the great resources that are avilable.
Well programming on the NintendoDS is possible, however you'll be breaking and stretching quite a few laws there.
However, I should say that I learned most of what I did about hardware doing stuff just like that while I was back in school. I learned a LOT from doing that sort of stuff ;)
But I wouldn't recommend it to youngsters or newcomers because you'll be totally out of support (lots of frustration), might break your game console, and unless you already know about programming the learning curve is WAY to high.
Might I recommend starting out with flash or PC games before console programming?
Edit: When I mean breaking/stretching laws, I don't mean writing code for your hardware, that's up to you. But it is illegal to buy pirate memory cards and card writers (that infringe on patent laws). Also funding hardware piracy will unlikely be something to proud of.
Edit:#Mike F did you read my post? I said, I HAVE done this while I was a student, and I learned a lot. But it's still piracy, and yes I did my fair share of piracy when I was a poor student, but it's not something I'd want to teach someone just getting into programming. Would you?
Besides there are plenty of perfectly great ways to learn game programming without needing to hack stuff like RPGMaker, XNA, etc...
#Mike F: Once more just for the sake of it NDS flash-roms are produced through patent piracy(/infringement), not software piracy, as in it is illegal to produce such hardware because it goes against patent laws. And the companies that make such flash rom hardware are piracy companies that traffic their goods in the grey market, Its as simple as that. I'm not talking about "software piracy" at all here. Thats why I mentioned twice already that anyone is free to run whatever software on their hardware, be homebrew or whatever.
Ars Technica just came out w/ an article about open-source gaming consoles. They are hand-helds, so I don't know if that's an issue but they seem to be pretty nice, with lots of features to tinker with.