The following code works as expected.
import boto.ec2
conn = boto.ec2.connect_to_region("us-east-1", aws_access_key_id='xxx', aws_secret_access_key='zzz')
sg = conn.create_security_group('test_delete', 'description')
auth = conn.authorize_security_group(sg.name, None, None, ip_protocol='tcp', from_port='22', to_port='22', cidr_ip='0.0.0.0/0')
I can select "All traffic" option from user interface. There is no equivalent here in boto.
I am aware of the security risks involved, but for some reason I want to open all ports (to / from) for all traffic using boto.
use 'IpProtocol': '-1' for "All traffic" option, see below code for details.
def create_ingress_rules (credentials=None,securitygroupid=None, region_name=None):
print("3-Start creating ingress rule(s)...")
create_ingress_rules_handler = \
boto3.client('ec2',
aws_access_key_id=credentials['AccessKeyId'],
aws_secret_access_key=credentials['SecretAccessKey'],
aws_session_token=credentials['SessionToken'],
region_name=region_name)
try:
data = create_ingress_rules_handler.authorize_security_group_ingress(
GroupId=securitygroupid,
IpPermissions=[
{'IpProtocol': '-1',
'FromPort': 0,
'ToPort': 65535,
'IpRanges': [{'CidrIp': '0.0.0.0/0','Description': 'Temporary inbound rule for Guardrail Testing'}]}
])
print('Complete creating Ingress rule...')
except ClientError as e:
print(e)
I think you just have to specify the min and max values for a port number. Since it is a 16-bit value, the value can range from 0 to 65535. So:
auth = conn.authorize_security_group(sg.name, None, None, ip_protocol='tcp', from_port=0, to_port=65535, cidr_ip='0.0.0.0/0')
Should allow traffic on all ports for the TCP protocol.
Related
This question already has an answer here:
Is there a way to store Mosquitto payload into an MySQL database for history purpose?
(1 answer)
Closed 4 years ago.
I've connected a device that communicates to my mosquitto MQTT server (RPi) and is sending out publications to a specified topic. What I want to do now is to store the messages published on that topic on the MQTT server into a MySQL database. I know how MySQL works, but I don't know how to listen for these incoming publications. I'm looking for a light-weight solution that runs in the background. Any pointers or ideas on libraries to use are very welcome.
I've done something similar in the last days:
live-collecting weatherstation-data with pywws
publishing with pywws.service.mqtt to mqtt-Broker
python-script on NAS collecting the data and writing to MariaDB
#!/usr/bin/python -u
import mysql.connector as mariadb
import paho.mqtt.client as mqtt
import ssl
mariadb_connection = mariadb.connect(user='USER', password='PW', database='MYDB')
cursor = mariadb_connection.cursor()
# MQTT Settings
MQTT_Broker = "192.XXX.XXX.XXX"
MQTT_Port = 8883
Keep_Alive_Interval = 60
MQTT_Topic = "/weather/pywws/#"
# Subscribe
def on_connect(client, userdata, flags, rc):
mqttc.subscribe(MQTT_Topic, 0)
def on_message(mosq, obj, msg):
# Prepare Data, separate columns and values
msg_clear = msg.payload.translate(None, '{}""').split(", ")
msg_dict = {}
for i in range(0, len(msg_clear)):
msg_dict[msg_clear[i].split(": ")[0]] = msg_clear[i].split(": ")[1]
# Prepare dynamic sql-statement
placeholders = ', '.join(['%s'] * len(msg_dict))
columns = ', '.join(msg_dict.keys())
sql = "INSERT INTO pws ( %s ) VALUES ( %s )" % (columns, placeholders)
# Save Data into DB Table
try:
cursor.execute(sql, msg_dict.values())
except mariadb.Error as error:
print("Error: {}".format(error))
mariadb_connection.commit()
def on_subscribe(mosq, obj, mid, granted_qos):
pass
mqttc = mqtt.Client()
# Assign event callbacks
mqttc.on_message = on_message
mqttc.on_connect = on_connect
mqttc.on_subscribe = on_subscribe
# Connect
mqttc.tls_set(ca_certs="ca.crt", tls_version=ssl.PROTOCOL_TLSv1_2)
mqttc.connect(MQTT_Broker, int(MQTT_Port), int(Keep_Alive_Interval))
# Continue the network loop & close db-connection
mqttc.loop_forever()
mariadb_connection.close()
If you are familiar with Python the Paho MQTT library is simple, light on resources, and interfaces well with Mosquitto. To use it simply subscribe to the topic and set up a callback to pass the payload to MySQL using peewee as shown in this answer. Run the script in the background and call it good!
import os, sys
AWS_DIRECTORY = '/home/jenkins/.aws'
certificates_folder = 'my_folder'
SUCCESS = 'success'
class AmazonKMS(object):
def __init__(self):
# making sure boto3 has the certificates and region files
result = os.system('mkdir -p ' + AWS_DIRECTORY)
self._check_os_result(result)
result = os.system('cp ' + certificates_folder + 'kms_config ' + AWS_DIRECTORY + '/config')
self._check_os_result(result)
result = os.system('cp ' + certificates_folder + 'kms_credentials ' + AWS_DIRECTORY + '/credentials')
self._check_os_result(result)
# boto3 is the amazon client package
import boto3
self.kms_client = boto3.client('kms', region_name='us-east-1')
self.global_key_alias = 'alias/global'
self.global_key_id = None
def _check_os_result(self, result):
if result != 0 and raise_on_copy_error:
raise FAILED_COPY
def decrypt_text(self, encrypted_text):
response = self.kms_client.decrypt(
CiphertextBlob = encrypted_text
)
return response['Plaintext']
when using it
amazon_kms = AmazonKMS()
amazon_kms.decrypt_text(blob_password)
getting
E ClientError: An error occurred (AccessDeniedException) when calling the Decrypt operation: The ciphertext refers to a customer master key that does not exist, does not exist in this region, or you are not allowed to access.
stacktrace is
../keys_management/amazon_kms.py:77: in decrypt_text
CiphertextBlob = encrypted_text
/home/jenkins/.virtualenvs/global_tests/local/lib/python2.7/site-packages/botocore/client.py:253: in _api_call
return self._make_api_call(operation_name, kwargs)
/home/jenkins/.virtualenvs/global_tests/local/lib/python2.7/site-packages/botocore/client.py:557: in _make_api_call
raise error_class(parsed_response, operation_name)
This happens in a script that runs once an hour.
it's only failing 2 -3 times a day.
after a retry it succeed.
Tried to upgraded from boto3 1.2.3 to 1.4.4
what is the possible cause for this behavior ?
My guess is that the issue is not in anything you described here.
Most likely the login-tokes time out or something along those lines. To investigate this further a closer look on the way the login works here is probably helpful.
How does this code run? Is it running inside AWS like on Lambda or EC2? Do you run it from your own server (looks like it runs on jenkins)? How is the login access established? What are those kms_credentials used for and how do they look like? Do you do something like assumeing a role (which would probably work through access tokens which after some time will no longer work)?
Here I am trying to use a plugin to check whether the service running or not, if there is any warning or any critical action required, at the same time the performance parameter.
We have used below plugin to check if a server is alive or not and read it's performance data JSON
https://github.com/drewkerrigan/nagios-http-json
I am trying to read a JSON file as below which is hosted on http://localhost:8080/sample.json
The plugin works perfectly on Command line, it shows me all the Metrics available.
$:/usr/lib/nagios/plugins$ ./check_http_json.py -H localhost:8080 -p sample.json -m metrics.etp_count metrics.atc_count
OK: Status OK.|'metrics.etp_count'=101 'metrics.atc_count'=0
But when I try the same in Icinga2 configuration, it doesn't show me this performance metrics, although it doesn't give any error but at the same time it don't show any value.
find the JSON, Command.conf and Service.conf as follows.
{
"metrics": {
"etp_count": "0",
"atc_count": "101",
"mean_time": -1.0,
}
}
Below are my commands.conf and services.conf
commands.conf
/* Json Read Command */
object CheckCommand "json_check"{
import "plugin-check-command"
command = [PluginDir + "/check_http_json.py"]
arguments = {
"-H" = "$server_port$"
"-p" = "$json_path$"
"-w" = "$warning_value$"
"-c" = "$critical_value$"
"-m" = "$Metrics1$,$Metrics2$"
}
}
services.conf
apply Service "json"{
import "generic-service"
check_command = "json_check"
vars.server_port="localhost:8080"
vars.json_path="sample.json"
vars.warning_value="metrics.etp_count,1:100"
vars.critical_value="metrics.etp_count,101:1000"
vars.Metrics1="metrics.etp_count"
vars.Metrics2="metrics.atc_count"
assign where host.name == NodeName
}
Does any one have any idea how can we pass multiple values in Command.conf and Service.conf??
I have resolved the issue.
I had to change the Plugin file "check_http_json.py" for below code
def checkMetrics(self):
"""Return a Nagios specific performance metrics string given keys and parameter definitions"""
metrics = ''
warning = ''
critical = ''
if self.rules.metric_list != None:
for metric in self.rules.metric_list:
Replaced With
def checkMetrics(self):
"""Return a Nagios specific performance metrics string given keys and parameter definitions"""
metrics = ''
warning = ''
critical = ''
if self.rules.metric_list != None:
for metric in self.rules.metric_list[0].split():
Actually the issue was the list was not handled properly, so it was not able to iterate through the items in the list, it was considering it as a single string due to services.config file.
it had to be further get split to get the items in the Metrics string.
I compiled Bind 9 from source (see below) and set up Bind9 with MySQL DLZ.
I keep getting an error when I attempt to fetch anything from the server about buffer overflow. I've googled many times but can not find anything on how to fix this error.
Configure options:
root#anacrusis:/opt/bind9/bind-9.9.1-P3# named -V BIND 9.9.1-P3 built
with '--prefix=/opt/bind9' '--mandir=/opt/bind9/man'
'--infodir=/opt/bind9/info' '--sysconfdir=/opt/bind9/config'
'--localstatedir=/opt/bind9/var' '--enable-threads'
'--enable-largefile' '--with-libtool' '--enable-shared'
'--enable-static' '--with-openssl=/usr' '--with-gssapi=/usr'
'--with-gnu-ld' '--with-dlz-postgres=no' '--with-dlz-mysql=yes'
'--with-dlz-bdb=no' '--with-dlz-filesystem=yes' '--with-dlz-stub=yes'
'--with-dlz-ldap=yes' '--enable-ipv6' 'CFLAGS=-fno-strict-aliasing
-DDIG_SIGCHASE -O2' 'LDFLAGS=-Wl,-Bsymbolic-functions' 'CPPFLAGS=' using OpenSSL version: OpenSSL 1.0.1 14 Mar 2012 using libxml2
version: 2.7.8
This is the error I get when I dig example.com (with debug):
Query String: select ttl, type, mx_priority, case when
lower(type)='txt' then concat('"', data, '"')
else data end from dns_records where zone = 'example.com' and host = '#'
17-Sep-2012 01:09:33.610 dns_rdata_fromtext: buffer-0x7f5bfca73360:1:
unexpected end of input 17-Sep-2012 01:09:33.610 dns_sdlz_putrr
returned error. Error code was: unexpected end of input 17-Sep-2012
01:09:33.610 Query String: select ttl, type, mx_priority, case when
lower(type)='txt' then concat('"', data, '"')
else data end from dns_records where zone = 'example.com' and host = '*'
17-Sep-2012 01:09:33.610 query.c:2579: fatal error: 17-Sep-2012
01:09:33.610 RUNTIME_CHECK(result == 0) failed 17-Sep-2012
01:09:33.610 exiting (due to fatal error in library)
Named.conf
options {
directory "/opt/bind9/";
allow-query-cache { none; };
allow-query { any; };
recursion no;
};
dlz "Mysql zone" {
database "mysql
{host=localhost dbname=system ssl=false user=root pass=*password*}
{select zone from dns_records where zone = '$zone$'}
{select ttl, type, mx_priority, case when lower(type)='txt' then concat('\"', data, '\"')
else data end from dns_records where zone = '$zone$' and host = '$record$'}
{}
{}
{}
{}";
};
Do you run named single-threaded (with "-n 1" parameter)? If not, named will crash in various places when working on more than one query in parallel, since the MySQL DLZ module is not thread safe.
Manually log into the DB and run the query. See what it comes up with. The error says it's got an unexpected end of input, meaning it was expecting to get something and it never got it. So the first thing is to see if it you can get it manually. Maybe the mysqld isn't running. Maybe the user isn't defined or password is set wrong or permissions are not granted on that table. These could all account for the error.
Assuming all this works then you have two options: Enable more logging in your named.conf so you have more data to work with on what's happeningRemove and reinstall BIND, ensuring that all hashes match on all libraries and that all dependancies are in place.
I have gotten Bind with DLZ working on CentOS 7. I do not get the error that is effecting you.
I realize this is an older post but I thought I would share my conf files , and configure options.
I am using Bind 9.11.0
configure
./configure --prefix=/usr --sysconfdir=/etc/bind --localstatedir=/var --mandir=/usr/share/man --infodir=/usr/share/info --enable-threads --enable-largefile --with-libtool --enable-shared --enable-static --with-openssl=/usr --with-gssapi=/usr --with-gnu-ld --with-dlz-postgres=no --with-dlz-mysql=yes --with-dlz-bdb=no --with-dlz-filesystem=yes --with-dlz-stub=yes --enable-ipv6
named.conf
// This is the primary configuration file for the BIND DNS server named.
//
// Please read /usr/share/doc/bind9/README.Debian.gz for information on the
// structure of BIND configuration files in Debian, *BEFORE* you customize
// this configuration file.
//
// If you are just adding zones, please do that in /etc/bind/named.conf.local
#auskommentiert !!!
#include "/etc/bind/named.conf.options";
#include "/etc/bind/named.conf.local";
#include "/etc/bind/named.conf.default-zones";
key "rndc-key" {
// how was key encoded
algorithm hmac-md5;
// what is the pass-phrase for the key
secret "noway";
};
#options {
#default-key "rndc-key";
#default-server 127.0.0.1;
#default-port 953;
#};
controls {
inet * port 953 allow { any; } keys { "rndc-key"; };
#inet * port 53 allow { any; } keys { "rndc-key"; };
};
logging {
channel query.log {
file "/var/log/query.log";
// Set the severity to dynamic to see all the debug messages.
severity dynamic;
};
category queries { query.log; };
};
dlz "Mysql zone" {
database "mysql
{host=172.16.254.100 port=3306 dbname=dyn_server_db user=db_user pass=db_password}
{SELECT zone FROM dyn_dns_records WHERE zone = '$zone$'}
{SELECT ttl, type, mx_priority, IF(type = 'TXT', CONCAT('\"',data,'\"'), data) AS data
FROM dyn_dns_records
WHERE zone = '$zone$' AND host = '$record$' AND type <> 'SOA' AND type <> 'NS'}
{SELECT ttl, type, data, primary_ns, resp_person, serial, refresh, retry, expire, minimum
FROM dyn_dns_records
WHERE zone = '$zone$' AND (type = 'SOA' OR type='NS')}
{SELECT ttl, type, host, mx_priority, IF(type = 'TXT', CONCAT('\"',data,'\"'), data) AS data, resp_person, serial, refresh, retry, expire, minimum
FROM dyn_dns_records
WHERE zone = '$zone$' AND type <> 'SOA' AND type <> 'NS'}
{SELECT zone FROM xfr_table where zone='$zone$' AND client = '$client$'}";
};
I want to learn yii as my first framework. And I'm trying to make the contact form work. But I got this error:
I've already configured php.ini file from:
C:\wamp\bin\php\php5.3.0
And changed the default to these values:
[mail function]
; For Win32 only.
; http://php.net/smtp
SMTP = ssl:smtp.gmail.com
; http://php.net/smtp-port
smtp_port = 23
; For Win32 only.
; http://php.net/sendmail-from
sendmail_from = myemail#gmail.com
I've seen from here that gmail doesn't use port 25, which is the default in the php.ini. So I used 23. And also opened that port in the windows 7 firewall. Via inbound rules.
Then I also edited the main config in my yii application, to match the email that I'm using:
// application-level parameters that can be accessed
// using Yii::app()->params['paramName']
'params'=>array(
// this is used in contact page
'adminEmail'=>'myemail#gmail.com',
),
);
Finally, I restarted wampserver. Then cleared all my browsing data. Why then to I still see that its pointing out port 25 in the error. Have I miss something? Please help.
Heres a simple python script which could allow you to run a mail server on localhost, you dont have to change anything. Sorry if im a bit late.
import smtpd
import smtplib
import asyncore
class SMTPServer(smtpd.SMTPServer):
def __init__(*args, **kwargs):
print "Running fake smtp server on port 25"
smtpd.SMTPServer.__init__(*args, **kwargs)
def process_message(*args, **kwargs):
to = args[3][0]
msg = args[4]
gmail_user = 'yourgmailhere'
gmail_pwd = 'yourgmailpassword'
smtpserver = smtplib.SMTP("smtp.gmail.com",587)
smtpserver.ehlo()
smtpserver.starttls()
smtpserver.ehlo
smtpserver.login(gmail_user, gmail_pwd)
smtpserver.sendmail(gmail_user, to, msg)
print 'sent to '+to
pass
if __name__ == "__main__":
smtp_server = SMTPServer(('localhost', 25), None)
try:
asyncore.loop()
except KeyboardInterrupt:
smtp_server.close()
#end of code
Note: I used args[3][0] and args[4] as to address and message as the args sent by my php mail() corresponded to an array of args[3][0] as receipent email
If you open the php.ini file in WAMP, you will find these two lines:
smtp_server
smtp_port
Add the server and port number for your host (you may need to contact them for details)
The following two lines don't exist by default:
auth_username
auth_password
So you will need to add them to be able to send mail from a server that requires authentication. So an example may be:
smtp_server = mail.example.com
smtp_port = 25
auth_username = example_username#example.com
auth_password = example_password
ps: you should not use your personal mail here. for an obvious reason.
If using WAMP, the php.ini to be configured is present in the wamp/bin/apache/Apache_x_y/bin folder
where _x_y is related to the version of the Apache build used by your wamp installation
uncomment extension=php_openssl.dll at php.ini in WAMP server ("D:\wamp\bin\apache\Apache2.4.4\bin\php.ini")
In the file "D:\wamp\www\mantisbt-1.2.15\config_inc.php"
# --- Email Configuration ---
$g_phpMailer_method = PHPMAILER_METHOD_SMTP;
$g_smtp_host = 'smtp.gmail.com';
$g_smtp_connection_mode = 'ssl';
$g_smtp_port = 465;
$g_smtp_username = 'yourmail#gmail.com';
$g_smtp_password = 'yourpwd';
$g_enable_email_notification = ON;
$g_log_level = LOG_EMAIL | LOG_EMAIL_RECIPIENT;
$g_log_destination = 'file:/tmp/log/mantisbt.log';
$g_administrator_email = 'administrator#example.com';
$g_webmaster_email = 'webmaster#example.com';
$g_from_email = 'noreply#example.com';
$g_return_path_email = 'admin#example.com';
$g_from_name = 'Mantis Bug Tracker';
$g_email_receive_own = OFF;
$g_email_send_using_cronjob = OFF;