I'm writing a program that manages data entered by users. I plan to open a test version to the public and have no idea how many users there may be.
I want my program to test when memory is getting low so that I know when to buy more server space and so that I can automatically restrict data entry when necessary. What's a good way to detect memory shortage? Allocate garbage space temporarily to get the exception? Is there a better way?
This may be best accomplished outside of your application using a performance monitoring tool. Windows Server can be configured to do this for you; see this question. There are other tools out there that help you monitor your servers, and I advise you to use an existing system unless you absolutely have to do this with Python.
If you must absolutely do this using Python, then have a look at the psutil library:
psutil (python system and process utilities) is a cross-platform
library for retrieving information on running processes and system
utilization (CPU, memory, disks, network) in Python. It is useful
mainly for system monitoring, profiling and limiting process resources
and management of running processes. It implements many
functionalities offered by command line tools such as: ps, top, lsof,
netstat, ifconfig, who, df, kill, free, nice, ionice, iostat, iotop,
uptime, pidof, tty, taskset, pmap. It currently supports Linux,
Windows, OSX, FreeBSD and Sun Solaris, both 32-bit and 64-bit
architectures, with Python versions from 2.4 to 3.4. Pypi is also
known to work.
You may combine this with the email package to send the alerts.
Related
I'm currently developing a Python application and I would like to know if there are any ways to pack MongoDB and MySQL (or Postgresql) into the application. By packing I mean taking those application binaries and distribute them with the application files.
For example, Metasploit PRO has some applications like nginx, postgresql, java, ruby, etc... under /opt/metasploit (they come with the application setup), and I would like to know if that could be done with any Linux application. And if so, how could I "choose" what binaries are needed? Would they work for any Debian distro? Can any application follow that procedure? Could it be done with MySQL and MongoDB?
P.D: I would like to do this to distribute one unique application instead of having to "obligate" the user to setup the databases independently, and for pure curiosity.
Thank you very much in advance!
MongoDB already distributes its binaries as standalone binaries in the sense that everything needed for the database (or shell tools) to run is included in the respective binary (mongo/mongos/mongod).
However, these binaries are OS (Linux distribution)-specific. Meaning, for example, they dynamically link against libssl and libcurl and you need to have the right versions of those libraries on the host system. So, for example, a MongoDB binary for Ubuntu 14.04 is likely to not work on Ubuntu 16.04.
As far as I know MongoDB does not support building for "generic linux". Only specific OSes like Ubuntu 16.04 are supported.
With that said, you could possibly build a "portable" MongoDB yourself if you accept some limitations, since its source code is available:
You need to figure out how to build MongoDB on some linux distribution that gives you the baseline glibc that would be compatible with all of your targets.
You may have to forego functionality like TLS connections, or figure out how to link against openssl statically (probably non-trivial).
This would be easier with older MongoDB versions (4.0, 3.6) since they have fewer system dependencies.
I think you can pack the required services and your application as Docker image or Virtual Machine box.
As my experience, I used to package the MongoDB and other Linux CLI tools with my NodeJS web application into a VM box using Vagrant. Or you can use Docker if you prefer container-based application.
If you use Vagrant, the provisioning feature may help you to setup the database before running the application. Check https://www.vagrantup.com/docs/provisioning
I am beginner. I want install freebsd on VM and test open source world! I want to write a small function and to put it into kernel of OS, and then I want to use it in another program as a system function.
I just installed freebsd11 on VM. I know a command line environment! I should use a GUI? Where is kernel?
FreeBSD is a wonderful beast once you know all its capabilities, if you want to play with jails, ZFS and build your own kernels, probably this already build image can be a good starting point:
https://fabrik.red/post/test/ more info can be found here including scripts about how to create your own images/jails (custom kernel): https://github.com/fabrik-red/images
There is no GUI on the images, and maybe you will never need one, unless you want to use FreeBSD as a desktop, but FreeBSD shines more on headless systems (no GUI).
Update:
For GUI probably you can give a try to trueos.org or you can install your own Xorg, desk environments
Sounds like you should learn a bit more about what the operating system does before trying to modify the kernel. There are lots of resources about the unix kernels. See the developers handbook, https://www.freebsd.org/doc/en/books/developers-handbook/ or just google and you will find many resources.
A project with the following technologies and components has surfaced: to up a Web stack solution initially composed with Ubuntu, JDK, JBoss, Spring MVC 3.0+, and MySQL.
In planning this project, I have been struggling to find answers to the following questions for first steps, best practices, and sequence:
1) Does the JDK (and JBoss) need to be installed as ‘root’? (I have seen articles that mention it is not a good idea to operate in root unless absolutely necessary due to the fatal consequences.)
2) Does Ubuntu need to be installed as a Server in order to accomplish all this, or can it also be installed as a Desktop? I have not been able to determine if having a JBoss and MySQL need to be installed on top of Linux as a server.
3) Does Maven need to be used within Spring STS in order to get JBoss, and MySQL (and in the future Hibernate) to work successfully together?
4) My intent is to install in this order: a) Ubuntu -> b) Java -> c) JBoss -> d) Spring STS -> and e) MySQL. Are there any blatant conflicts in this sequence?
JBoss will require Java (recommend Java 7) before it will do anything. I don't think it really "installs" per-say, but rather just unpacks to some directory (even if you install from the package manager, it just really extracts itself). I question your need for Spring since JBoss and Java EE in general really does everything Spring does, and better now-a-days. Unless you have a specific requirement for Spring, I'd question this extra dependency.
For linux - in a high level, any OS can be a "server", all it needs is to be capable of serving things (web pages, ssh connections, etc). In M$ world, different "levels" of the OS have been specially designed based on anticipated task/workload. So for example, while Windows 7 can indeed run as a server, it was not designed for it and therefore may not be optimized or include helper utilities and tools to make life easier as a sys admin of the system. Windows Server on the other hand does include all the "normal" server tools and lots of goodies to make maintaining and setting the server up easier.
In linux land, this is no such thing. Linux is the kernel that talks back and forth with the bare metal, etc... and Distro makers will take that and build an OS around the kernel, basically just attaching any packages they feel their distro needs... such as wget, or cat, or any other standard userland apps, plus some non-standard such as mysql or java or whatever they want.
Now, some distributions of linux will tailor themselves at being "server" ready, while others will tailor themselves at being a desktop OS. The difference? It's really just whatever default packages the distribution maker decides to include or not. For example, the overwhelming majority of linux servers are run completely headless, and therefore there is absolutely no reason to have X11 and a huge bloated GUI environment installed and/or running on that system... it's pointless. Also, an "average joe" user does not need MySQL installed by default on his desktop system since it would only bloat his system and he likely won't ever use it.
So basically it comes down to default installed packages.
Some linux server distros take this further and exercise extreme caution when making updates, patches, or new releases in the name of stability and security, while on the other hand most desktop distros are more haphazard with their updates since if it breaks a home users web browser, it's probably not a huge deal... but if a server update breaks the webserver application stack, now that's a serious problem.
So you'll find server OS's like CentOS (based on upstream RHEL) are extremely slow to bring in the "latest and greatest" features that desktop OS's get early on. Their goal is high security and long term stability.
Now, for Ubuntu. While I certainty know a lot of folks run Ubuntu as their server OS choice (partly due to Amazon choosing Ubuntu as the default linux VM for their ECS cloud), but I'd really question this. Ubuntu is not focused on being a server. It's focused on being a great all-around desktop oriented OS. Yes the LTS version is meant for long term stability, but it's based out of a desktop OS, so it's still not the focus.
IMHO, I'd go with CentOS because it's free and completely binary compatible version of RHEL - and RHEL is the de-facto standard for enterprise-grade linux servers. Be aware though, the RHEL way of doing things is a bit different than the debian way -- so there is no apt-get, you must use yum install instead. Startup scripts are different and some ways of doing things are different, but really, once you know linux, you know linux.
EDIT: Also check into Jenkins - its a free opensource continuous integration system that runs on JBoss or Tomcat or any other container, and can automagically pull your code from a repo (github, git, svn, etc) and compile/package it then push it to live deployment. You setup your ANT or Maven build scripts, and it can kick off on a schedule or however you configure it.
EDIT EDIT: I'd also recommend using OpenJDK -- as it's likely included in your package manager (for just about every disto) and will be more updated than the oracle version if it's in your package manager too. I've found most "server" distros will have OpenJDK 7 while only having Oracle java 6 in their package managers. Also, installing it via the package manager will enable you to keep it updated a ton easier.
Installed as root, why not? Run as root, probably not a good idea.
If you want a desktop, install a desktop distrib. If you want a server, install a server distrib. This doesn't change what can and can't be run in the OS. It only changes what is installed by default.
Maven is a build tool. JBoss doesn't care how you build your app. All it cares about is if the application you deploy is a valid Java EE application.
No. You need an OS, so Ubuntu must come first. JBoss and (AFAIK) Spring STS need a JRE to run, as they're Java applications, so Java should be installed before them. MySQL is independent of JBoss, STS and Java, so you can install it whenever you want.
Note that if you're struggling just with this installation part, be prepared to suffer with the rest. Building a Java EE webapp is not a piece of cake, and you should probably find some experienced developer to help you, as it seems you're only beginning with Java.
I am a fresh man in kvm,qemu-kvm and kvm are both very complicated now.
Anyone can introduce some primers about qemu-kvm and kvm?
thanks very much!
KVM stands for kernel based virtual machine. it enables you to create as many number of virtual machine as you like. These machine can be of two types LVM based or Non-LVM based.
Those machine which are LVM based you can take live backup for them. for non-lvm based VM you cannot take live backup i.e. they will be paused when you take backup for them. please refer KVM home page KVM Home Page.
QEMU is a generic and open source machine emulator and virtualizer.When used as a machine emulator, QEMU can run OSes and programs made for one machine (e.g. an ARM board) on a different machine (e.g. your own PC). By using dynamic translation, it achieves very good performance. When used as a virtualizer, QEMU achieves near native performances by executing the guest code directly on the host CPU. QEMU supports virtualization when executing under the Xen hypervisor or using the KVM kernel module in Linux. When using KVM, QEMU can virtualize x86, server and embedded PowerPC, and S390 guests.
For managing the KVM VM's you need to install Libvirt which is the virtualization library. It provides you the tools for starting, suspending, resuming, cloning, restarting, listing of virtual machine. Please refer Libvirt home page for more reference.
If you are working on some backup or recovery process then I suggest you to go through this excellent perl script as well it will give a fair idea of how the backup and snapshot is being taken for KVM VM's.
KVM based virtual machines are not complicated once you go through the theory of them and start implementing them. I believe once you start working on them you will find fun in managing them.
Putting in a nutshell
QEMU : An emulator which translates the instruction of guest operating system to host operating system. As you can guess that translation has a certain cost, you will not see Guest machine working as fast as host machine.
For more info see the QEMU wiki
KVM (Kernal Virtual Machine): A module in Kernel which support Virtual Machine (host operating system) in hardware. By support I mean that if your guest architecture is same as host architecture, then certainly there is no need to translate the instructions as they can directly be executed by host. For this modern hardware are equipped with special registers and storage location which is leveraged by KVM. Also KVM is a module, some driver is needed to use the KVM, which is qemu also.
For more info see the KVM section in the same wiki.
QEMU-KVM : As I above mentioned, KVM is a module only, qemu is needed (or other) to use KVM. When KVM is used with QEMU, control transfers from QEMU to KVM and vice-versa over the execution.
Talking about KVM is talking about virtualization technology or about kernel modules (kvm.ko, kvm-intel.ko or kvm-amd-ko). Sometimes KVM is mentioned as a virtual machine, this is not correct, because KVM does not provide virtualized hardware.
Source
I have access to a server machine, with 3 CUDA enabled GPUs in it, and I would like to use NVidia Parallel Nsight, to remotly debug on the machine.
This works just find.
Now, is it possibble, to start another debug session (possibbly by another developer), on the same machine, but on another GPGPU?
Is it possibble, to do this, if I use gdb on linux?
Thanks,
krisy
Krisy, yes this is possible.
However this case/scenario that you mentioned has not been actively tested internally by the Nsight team yet. I tried this our real quick on a system with a similar setup as the one you mentioned and I was able to debug 2 different instances of CUDA app simulataneously (provided each app runs on a different unique device that is not connected to any output display).
The stability of this is not guaranteed. From what I've tried so far, this worked for me and it should work in theory as well but there were instances where I experienced sluggish behavior on my system.
For other developers who are interested to know more about this, please take a look at: http://forums.nvidia.com/index.php?showtopic=201211