Building Custom Platform on ElasticBeanstalk Packer Error - amazon-elastic-beanstalk

I am trying to build a custom platform using the provided example (NodePlatform_Ubuntu) at
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/custom-platforms.html#custom-platforms-pda
I installed EB CLI (my OS is MacOSX El Captain):
➜ custom-platform ebp --version
EB CLI 3.10.1 (Python 2.7.1)
For some reason, EB CLI insists on using Python 2.7.1 instead of the already installed Python 3.x (python3 command works), but this shouldn't matter according to http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb-cli3-install.html it supports Python 2.7
While "ebp init" works perfectly. the "ebp create" fails.
➜ custom-platform ebp create
Creating application version archive "app-170427_145319".
Uploading custom-platform/app-170427_145319.zip to S3. This may take a while.
Upload Complete.
Note: An environment called 'eb-custom-platform-builder-packer' has been created in order to build your application. This environment will not automatically be terminated and it does have a cost associated with it. Once your platform creation has completed you can terminate this builder environment using the command 'eb terminate'.
INFO: createPlatform is starting.
INFO: Initiated platform version creation for 'custom-platform/1.0.2'.
INFO: Creating Packer builder environment 'eb-custom-platform-builder-packer'.
ERROR: Packer environment eb-custom-platform-builder-packer is not available, current status: terminated.
INFO: Failed to create platform version 'custom-platform/1.0.2'.
This is the error:
ERROR: Packer environment eb-custom-platform-builder-packer is not available, current status: terminated.
More logs (debug mode):
2017-04-27 14:19:55,698 (DEBUG) ebcli.lib.aws : Response: {u'Events': [{u'PlatformArn': 'arn:aws:elasticbeanstalk:us-west-2:107875334514:platform/custom-platform/1.0.1', u'Message': "Failed to create platform version 'custom-platform/1.0.1'.", u'Severity': 'INFO', u'EventDate': datetime.datetime(2017, 4, 27, 11, 19, 52, 84000, tzinfo=tzutc()), u'RequestId': 'f762956d-2b3a-11e7-8cd7-c96ae1e26915'}, {u'PlatformArn': 'arn:aws:elasticbeanstalk:us-west-2:107875334514:platform/custom-platform/1.0.1', u'Message': 'Packer environment eb-custom-platform-builder-packer is not available, current status: terminated.', u'Severity': 'ERROR', u'EventDate': datetime.datetime(2017, 4, 27, 11, 19, 51, 861000, tzinfo=tzutc()), u'RequestId': 'f762956d-2b3a-11e7-8cd7-c96ae1e26915'}], 'ResponseMetadata': {'date': 'Thu, 27 Apr 2017 11:19:52 GMT', 'RetryAttempts': 0, 'HTTPStatusCode': 200, 'RequestId': '6f7dcea9-2b3b-11e7-8cd7-c96ae1e26915'}}
ERROR: Packer environment eb-custom-platform-builder-packer is not available, current status: terminated.
INFO: Failed to create platform version 'custom-platform/1.0.1'.
Note that the app-170427_145319.zip is successfully uploaded to S3.
Any idea what's wrong?

It is was failing with instance profile(role) aws-elasticbeanstalk-custom-platform-ec2-role not getting necessary role inline policy attached to it.
Because of lot of modifications and changes, apparently due to an EB CLI bug, the instance profile got corrupted and was not taking InstanceProfile ARN when tried to create a profile with same name "aws-elasticbeanstalk-custom-platform-ec2-role".
So, we went ahead and created a new one named "custom-platform" and attached necessary inline policy permissions I have mentioned in the case.
Once that IAM issues are sorted, we went onto packer bundle,did "ebp init" and then ran "ebp create" but with -ip flag and used this instance profile custom-platform, as in
ebp create -ip custom-platform

Related

aws coldbuild failed: ebcli.objects.exceptions.NotFoundError: Environment "flask-daniel-test" not Found

I have github repo here https://github.com/DanielWu1978/flaskbeanstalk
And then configured a codebuild for it. When I run the following command from command line locally, it is ok, but when call the same code from codebuild, it raises the error of "ebcli.objects.exceptions.NotFoundError: Environment "flask-daniel-test" not Found.". Why this happens?
eb deploy flask-daniel-test --verbose
WARNING: Beanstalk configuration header 'eb_codebuild_settings' is missing from Buildspec file; will not use Beanstalk Code Build integration
INFO: Deploying code to flask-daniel-test in region us-west-2
INFO: Getting version label from git with git-describe
Creating application version archive "app-38c0-220705_182941662302".
INFO: creating zip using git archive HEAD
INFO: git archive output: .elasticbeanstalk/
.elasticbeanstalk/config.yml
.gitignore
Makefile
README.md
application.py
buildspec.yml
requirements.txt
INFO: Uploading archive to s3 location: flask-daniel/app-38c0-220705_182941662302.zip
Uploading flask-daniel/app-38c0-220705_182941662302.zip to S3. This may take a while.
Upload Complete.
INFO: Creating AppVersion app-38c0-220705_182941662302
2022-07-05 10:29:47 INFO Environment update is starting.
2022-07-05 10:29:51 INFO Deploying new version to instance(s).
2022-07-05 10:29:55 INFO Instance deployment successfully generated a 'Procfile'.
2022-07-05 10:30:04 INFO Instance deployment completed successfully.
2022-07-05 10:30:09 INFO New application version was deployed to running EC2 instances.
2022-07-05 10:30:09 INFO Environment update completed successfully.
But when run from codebuild. it has the following:

Gnome Boxes on Fedora 33 fails to open

I attempt to load gnome-boxes from the terminal (I'm running Fedora 33) and get the following error
$ gnome-boxes
(gnome-boxes:3194): Gtk-WARNING **: 12:34:57.343: GtkFlowBox with a model will ignore sort and filter functions
(gnome-boxes:3194): Gtk-WARNING **: 12:34:57.344: GtkListBox with a model will ignore sort and filter functions
(gnome-boxes:3194): Boxes-WARNING **: 12:34:57.904: libvirt-machine.vala:83: Failed to disable 3D Acceleration
(gnome-boxes:3194): Boxes-WARNING **: 12:34:57.913: libvirt-broker.vala:70: Failed to update domain 'fedora33-wor-2': Failed to set domain configuration: XML error: Invalid PCI address 0000:04:00.0. slot must be >= 1
(gnome-boxes:3194): Boxes-CRITICAL **: 12:34:57.916: boxes_vm_importer_get_source_media: assertion 'self != NULL' failed
Segmentation fault (core dumped)
My system:
$uname -a
Linux localhost.localdomain 5.9.16-200.fc33.x86_64 #1 SMP Mon Dec 21 14:08:22 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
I don't whether it's related but I recently updated from kernel 5.9.11 directly to 5.9.16 (haven't used the PC in question for some weeks) and before gnome-boxes was working as normal.
Please advise how I can restore gnome-boxes - I have some virtual machines that I need to access...
I faced this issue when I force stopped Gnome-Boxes while cloning a VM.
Deleting the conflicting VM will resolve your issue(in your case 'fedora33-wor-2').
To delete the VM in fedora, install "libvirt-client" which provides "virsh" using the command
dnf install libvirt-client
then double check the available VM's using
virsh list --all
Delete the VM using command,
virsh undefine VM_Name
#channel-fun solved the problem of staring up gnome-boxes.
But the real problem is in cloning procedure. The XML describing the new machine is malformed.
virt-clone --original fedora33-ser --auto-clone
works properly.
I know this is an old thread, but I had the same problem recently.
I shut down gnome boxes whilst it was cloning a vm, and shutdown the machine.
I then couldn't open boxes, as it would just crash.
I was able to delete the VM itself, and then deleted the XML file associated with it.
To delete the VM itself, go to :
$HOME/.var/app/org.gnome.Boxes/data/gnome-boxes/images (which in my case is a symbolic link to a data drive)
and delete the VM with the name that you were cloning to (or safer, just move it somewhere).
To delete the XML file associated with it:
$HOME/.var/app/org.gnome.Boxes/config/libvirt/qemu/
and delete (or safer move) the file that is named VM_NAME.xml.
Then boxes should open ok, at least it worked for me.
Extending on Channel Fun's answer for Ubuntu repos the package is libvirt-clients (note the plural s):
sudo apt install libvirt-clients
Check the available VM's using:
virsh list --all
Delete the VM using:
virsh undefine VM_Name
If you receive the error:
error: Refusing to undefine while domain managed save image exists
Then you can explicitly remove that also using the --managed-save flag:
virsh undefine VM_Name --managed-save

Integrating Apache Superset and Apache Drill

I installed Apache Drill through the link in the Drill Documentation. Apache Drill works fine. I also installed and got Apache Superset running using docker. Superset also works totally fine.
But my goal is to integrate Superset and Drill together. The only tutorial I was able to find was a tutorial from Dataist. When following this tutorial they ask us to add a database.
Since I am running both Drill and Superset in my local machine they ask us to type drill+sadrill://localhost:8047/dfs/test?use_ssl=False as the SQLAlchemy URI. They ask us to test the connection by pressing test connection.
When pressing test connection I get an error message as follows.
ERROR: {"error": "Connection failed!\n\nThe error message returned was:\nCan't load plugin: sqlalchemy.dialects:drill.sadrill", "stacktrace": "Traceback (most recent call last):\n File \"/home/superset/superset/views/core.py\", line 1755, in testconn\n engine = database.get_sqla_engine(user_name=username)\n File \"/home/superset/superset/utils/core.py\", line 132, in __call__\n value = self.func(*args, **kwargs)\n File \"/home/superset/superset/models/core.py\", line 911, in get_sqla_engine\n return create_engine(url, **params)\n File \"/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/__init__.py\", line 435, in create_engine\n return strategy.create(*args, **kwargs)\n File \"/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py\", line 61, in create\n entrypoint = u._get_entrypoint()\n File \"/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/url.py\", line 172, in _get_entrypoint\n cls = registry.load(name)\n File \"/usr/local/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py\", line 240, in load\n \"Can't load plugin: %s:%s\" % (self.group, name)\nsqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:drill.sadrill\n"}
Can someone figure out why I am getting this error. And also if there are any other tutorials to follow which give a better idea on how to set up Drill and superset.
I have encountered a similar issue while trying to connect elasticsearch. I guess the docker image you used was amancevice/superset. This issue cause because your image is not using the latest SQLAlchemy, SQLAlchemy-Utils packages. Upgrade or reinstall these packages to fix the issue.
To uninstall:
pip uninstall SQLAlchemy
pip uninstall SQLAlchemy-Utils
To install again (latest version):
pip install SQLAlchemy
pip install SQLAlchemy-Utils
I have reported this issue here https://github.com/amancevice/docker-superset/issues/158 maybe it get fixed with the upcoming images.

Getting Assemble script failed error building application in Redhat Openshift

I'm new to Redhat OpenShift and trying to deploy node application (with angularjs + mysql) and running into build issues.
Using openshift console created node application and in advanced options pointed to the private repository and linked configured secret (ssh key to private repository).
My build is failing with "Assemble script failed". Pasting the logs as below (from console - obfuscated private keys and values).
Not sure if I'm missing some configurations. Appreciate help on this.
Cloning "ssh://username#bitbucket.org/username/my-app.git" ...
Commit: xxxxxxxxxxxxxxxxx (Fixed readme)
Author: Name <email>
Date: Wed Sep 6 19:50:59 2017 -0700
Pulling image "docker-
registry.default.svc:5000/openshift/nodejs#sha256:0000000000000" ...
---> Installing application source
---> Building your Node application from source
Current git config
url.https://github.com.insteadof=git#github.com:
url.https://.insteadof=ssh://
url.https://github.com.insteadof=ssh://git#github.com
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.url=ssh://username#bitbucket.org/username/my-app.git
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.master.remote=origin
branch.master.merge=refs/heads/master
---> Installing dependencies
---> Using 'npm install -s --only=production'
error: build error: non-zero (13) exit code from docker-
registry.default.svc:5000/openshift/nodejs#sha256:0000000000000
Pls note that my source code is hosted on private repository and per log above it appears openshift is able to access the repository and download the source code.
Thanks Graham for the pointer. I recreated the application and this time in advanced options on the web console selected 1 GB (from default of 500 mb) after which my build worked fine.

Using MySQL with dev_appserver (Google App Engine) and Google SQL Service

I am using Google App Engine and Google SQL Service, and would like to use the option
dev_appserver.py --mysql_user=username myapp
in order to use a local MySQL database for development purposes, and SQL Service in prod environment.
I have MySQLdb installed and working:
/usr/bin>>python
Python 2.7.2 (v2.7.2:8527427914a2, Jun 11 2011, 15:22:34)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import MySQLdb
>>>
However, when I run the dev_appserver command, I get the following error:
zipimporter('/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/MySQL_python-1.2.3-py2.7-macosx-10.6-intel.egg', 'MySQLdb/')
ERROR 2011-09-12 09:34:53,541 rdbms_mysqldb.py:90] The rdbms API is not available because the MySQLdb library could not be loaded.
I have verified that the MySQL_python-1.2.3-py2.7-macosx-10.6-intel.egg is in the location (/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/) and that it is accessible.
Any idea what might be causing this error?
Full stacktrace provided below for information:
/Users/eddieboyd>>dev_appserver.py --debug --mysql_user=mysql eddiehelloworld
Warning: You are using a Python runtime (2.7) that is more recent than the production runtime environment (2.5). Your application may use features that are not available in the production environment and may not work correctly when deployed to production.
INFO 2011-09-12 09:34:53,290 appengine_rpc.py:159] Server: appengine.google.com
INFO 2011-09-12 09:34:53,304 appcfg.py:449] Checking for updates to the SDK.
DEBUG 2011-09-12 09:34:53,305 appengine_rpc.py:364] Sending HTTPS request:
POST /api/updatecheck?release=1.5.3&timestamp=1311108376&api_versions=%5B%271%27%5D HTTPS/1.1
Host: appengine.google.com
X-appcfg-api-version: 1
Content-type: application/octet-stream
User-agent: appcfg_py/1.5.3 Darwin/10.8.0 Python/2.7.2.final.0
INFO 2011-09-12 09:34:53,465 appcfg.py:466] The SDK is up to date.
WARNING 2011-09-12 09:34:53,465 datastore_file_stub.py:512] Could not read datastore data from /var/folders/++/++71vE++6+0++4RjPqRgNE+0Eyo/-Tmp-/dev_appserver.datastore
INFO 2011-09-12 09:34:53,478 py_zipimport.py:148] zipimporter('/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/MySQL_python-1.2.3-py2.7-macosx-10.6-intel.egg', 'MySQLdb/')
ERROR 2011-09-12 09:34:53,541 rdbms_mysqldb.py:90] The rdbms API is not available because the MySQLdb library could not be loaded.
ERROR 2011-09-12 09:34:53,541 dev_appserver_main.py:638] <type 'exceptions.NotImplementedError'>: Unable to find the MySQLdb library. Please see the SDK documentation for installation instructions.
DEBUG 2011-09-12 09:34:53,543 dev_appserver_main.py:640] Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_main.py", line 635, in main
dev_appserver.SetupStubs(appinfo.application, **option_dict)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 4668, in SetupStubs
rdbms_mysqldb.connect(database='')
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/rdbms_mysqldb.py", line 96, in connect
'Unable to find the MySQLdb library. Please see the SDK '
NotImplementedError: Unable to find the MySQLdb library. Please see the SDK documentation for installation instructions.
Poor man's solution.
I added
"import MySQLdb"
in dev_appserver.py.
It works now.
Ensure to use the latest version of the AppEngine SDK which currently is the version 1.6.2.
After that install the package for python mysql support: python-mysqldb.
Maybe your system does not have the MySQL module required. On Fedora 16 64 bits, I solved this by installing the MySQL python module:
yum install MySQL-python