G-WAN and persistent MySQL connexion - mysql

me again with a little question about G-WAN and MySQL.
This script below works fine ...
My only problem is when MySQL went down. G-WAN script crash and G-WAN as well.
What is the nest way to keep a persistent MySQL connexion and handle MySQL downtime?
typedef struct {
MYSQL *conn;
} data_t;
int main(int argc, char *argv[])
{
u64 start = getus();
data_t **data = (data_t**)get_env(argv, US_SERVER_DATA);
xbuf_t *reply = get_reply(argv);
if(!data[0]) // first time: persistent pointer is uninitialized
{
data[0] = (data_t*)calloc(1, sizeof(data_t));
if(!data[0])
return 500; // out of memory
data[0]->conn = (MYSQL *)mysql_init(data[0]->conn);
if(! data[0]->conn) {
xbuf_xcat(reply, "MySQL Error");
return(200);
}
if(! mysql_real_connect(data[0]->conn, "localhost", "root", "willow", "test", NULL, NULL, 0)) {
return 500;
}
xbuf_cat(reply, "initialized data<br>");
}
// Do what we want here ...

Here my working script :
#pragma link "/usr/lib64/mysql/libmysqlclient.so"
#pragma include "/usr/include/mysql"
#include <mysql.h>
#include <string.h>
#include "gwan.h" // G-WAN exported functions
//static MYSQL *conn = NULL;
typedef struct {
MYSQL *conn;
} data_t;
int main(int argc, char *argv[])
{
u64 start = getus();
data_t **data = (data_t**)get_env(argv, US_SERVER_DATA);
xbuf_t *reply = get_reply(argv);
my_bool my_true = true;
if(!data[0]) // first time: persistent pointer is uninitialized
{
data[0] = (data_t*)calloc(1, sizeof(data_t));
if(!data[0])
return 500; // out of memory
data[0]->conn = (MYSQL *)mysql_init(data[0]->conn);
mysql_options(data[0]->conn, MYSQL_OPT_RECONNECT, &my_true);
if(! data[0]->conn) {
xbuf_xcat(reply, "MySQL Error");
return(200);
}
if(! mysql_real_connect(data[0]->conn, "localhost", "root", "willow", "test", 0, NULL, 0)) {
return 500;
}
xbuf_cat(reply, "initialized data<br>");
}
if(mysql_ping(data[0]->conn) > 0) return 500; // Prevent G-WAN to crash
mysql_query(data[0]->conn, "DELETE FROM example");
for(int i =1; i< 10; i++) {
char sql[1024];
s_snprintf(sql, 1023, "INSERT INTO example (id, name, age) VALUES(%d, 'Olivier', 33)", i);
mysql_query(data[0]->conn, sql);
}
int count = 0;
// Query Database
mysql_query(data[0]->conn, "SELECT id, name, age FROM example");
MYSQL_RES *result = mysql_store_result(data[0]->conn);
MYSQL_ROW *row;
while ((row = mysql_fetch_row(result))) {
count++;
xbuf_xcat(reply, ": %s : %s : %s", row[0], row[1], row[2]);
xbuf_xcat(reply, "<br/>");
}
mysql_free_result(result);
xbuf_xcat(reply, "<br>%llUmicro seconds : %d<br/>", (getus()-start), mysql_field_count(data[0]->conn));
xbuf_xcat(reply, "MySQL Client Version: %d", mysql_get_client_version());
return 200;
}
So now even when mysql is down, G-WAN is still alive, and the script works again when mysql is UP. Hope it will help someone else.

It be great if You do a bit of research before asking a question...
See -> http://dev.mysql.com/doc/refman/5.5/en/mysql-ping.html

when MySQL went down. G-WAN script crash and G-WAN as well
Since you only publish the code that creates the persistent connection to MySQL (and not the code that uses MySQL later), there's no way to see how your code manages to crash G-WAN.
But it is most likely that this is the MySQL driver library that is crashing when you are trying to use an invalid socket.
The solution here is obviously to test the validity of the socket (its connected state) via something like:
ioctl(fd,FIONREAD,&bytes_available);
...before you pass it to the MySQL client library.

Related

C - alternative using snprintf to prepare MySQL statements?

I've been tearing my hair out for a while on this one. The C code is called from a bash script, which loops through a command's output in a while loop and passes variables to the C script as args. It goes through a list and partitions data properly. I've been using the C MySQL api, and up until now everything has been relatively straight forward. It tries to run a SELECT(EXISTS) command to dictate whether to input a new row, or update an existing one.
I have typed the command into MySQL terminal and it works perfectly. I have even printf'd it and copied the command directly into the terminal. It works....
So why then, am I getting Syntax errors? I've tried escaping fields and input using backticks, single quotes and double quotes and I'm still getting this dumbounding error. I thought maybe it was something to do with the null space? But I'm at my witts end. Here's the code, any advice would be greatly appreciated :)
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <mysql/mysql.h>
const int MAXLEN = 100;
/* Compile with:
gcc db.connect.c `mysql_config --libs` -O1
for the best results
*/
/* Function definitions for later */
void finish_with_error(MYSQL *con);
int send_query(MYSQL *con, char query[MAXLEN]);
/* If any SQL commands fail, return an error message */
void finish_with_error(MYSQL *con)
{
fprintf(stderr, "%s\n", mysql_error(con));
mysql_close(con);
exit(1);
}
/* Helper function to send queries to MySQL database */
int send_query(MYSQL *con, char query[MAXLEN])
{
if (mysql_query(con, query)) {
finish_with_error(con);
}
return 0;
}
int main(int argc, char ** argv)
{
// Establish MySQL API connection, if not- fail with err
MYSQL *con = mysql_init(NULL);
if (con == NULL) {
finish_with_error(con);
}
// Connection string.
if (mysql_real_connect(con, "localhost", "user", "password",
NULL, 0, NULL, 0) == NULL){
finish_with_error(con);
}
if (argv[1] == NULL){
printf("No query passed, terminating script \n");
return 1;
}
if (argv[1] != NULL) {
if( strcmp( argv[1], "--help" ) == 0 ) {
printf("This program was created to interact with MySQL, by checking and updating live network stats\n");
printf("It has 2 parameters, an IP address to look in the database for and a value to update a field by, \
if that IP address is found. ");
printf("If the value is not found, the program will insert a new row.");
return 1;
}
// Works out how much memory to allocate to buffer for snprintf
// Originally cmd_len was 65- as this was the amount of bits needed by the address string.
// This was changed to MAXLEN to prevent SEGFAULTS and give the function breathing room.
size_t cmd_len = MAXLEN;
size_t param_len = sizeof(argv[2]);
size_t q_len = cmd_len + param_len;
// Allocates that memory to a buffer, referenced as query
char *query = malloc(sizeof(char) * q_len);
snprintf(query, q_len, "SELECT EXISTS(SELECT * FROM `analytics`.`live` WHERE `foreign_addr` = `%s`)", argv[1]);
printf("%s\n", query);
send_query(con, query);
free(query);
// Used to store the result of the MySQL select commands
MYSQL_RES *result = mysql_store_result(con);
if (result == NULL) {
finish_with_error(con);
}
// num_fields stores the number of fields, i and x are counters, answer is 1 or 0
int num_fields = mysql_num_fields(result);
int i = 0;
// Loops through each row in the answer statement.
// There will only be one row in the answer, which will be 1 or 0
// Basically, if the IP is found.
MYSQL_ROW row;
while ((row = mysql_fetch_row(result))){
for (i=0; i<num_fields; i++) {
// If the IP isn't in the table
if(!atoi(row[i]))
send_query(con, argv[1]);
// If the IP is already in the table
if(atoi(row[i])) {
snprintf(query, q_len, "UPDATE analytics.live SET count=count+1 WHERE foreign_addr = '%s'", argv[1]);
printf("%s\n", query);
free(query);
snprintf(query, q_len, "UPDATE analytics.live SET dat_sent = dat_sent + %s", argv[2]);
printf("%s\n", query);
free(query);
}
}
}
mysql_close(con);
return 1;
}
mysql_close(con);
return 0;
}

how do i publish the data on mqtt of the bme680 sensor using ESP32cam

I am using windows 10 and working on getting the gas readings from BME680 using esp32cam. I tried to add this:
#include <WiFi.h>
#include <PubSubClient.h>
#include <Wire.h>
//#include <Adafruit_BME280.h>
#include <Adafruit_Sensor.h>
#include "Adafruit_BME680.h"
#include <Arduino_JSON.h>
//uint64_t mac;
//uint32_t high;
//uint32_t low;
// Replace the next variables with your SSID/Password combination
const char* ssid = "";
const char* password = "";
// Add your MQTT Broker IP address, example:
//const char* mqtt_server = "";
const char* mqtt_server = "";
const char* mqtt_port = "";
const char* mqtt_user = "";
const char* mqtt_password = "";
WiFiClient espClient;
PubSubClient client(espClient);
long lastMsg = 0;
char msg[50];
int value = 0;
//uncomment the following lines if you're using SPI
#include <SPI.h>
#define BME_SCK 14
#define BME_MISO 12
#define BME_MOSI 13
#define BME_CS 15
//Adafruit_BME280 bme; // I2C
//Adafruit_BME280 bme(BME_CS); // hardware SPI
Adafruit_BME680 bme(BME_CS, BME_MOSI, BME_MISO, BME_SCK); // software SPI
float temperature = 0;
float humidity = 0;
float pressure = 0;
float gas = 0;
//Device ID
//uint64_t chip_id;
//chip_id=ESP.getEfuseMac();//The chip ID is essentially its MAC address(length: 6 bytes).
// LED Pin
const int ledPin = 4;
void setup() {
Serial.begin(115200);
// default settings
// (you can also pass in a Wire library object like &Wire2)
//status = bme.begin();
if (!bme.begin(0x76)) {
Serial.println("Could not find a valid BME280 sensor, check wiring!");
while (1);
WiFi.begin();
}
setup_wifi();
client.setServer(mqtt_server,...);
client.setCallback(callback);
pinMode(ledPin, OUTPUT);
// Set up oversampling and filter initialization
bme.setTemperatureOversampling(BME680_OS_8X);
bme.setHumidityOversampling(BME680_OS_2X);
bme.setPressureOversampling(BME680_OS_4X);
bme.setIIRFilterSize(BME680_FILTER_SIZE_3);
bme.setGasHeater(320, 150); // 320*C for 150 ms
}
void setup_wifi() {
delay(10);
// We start by connecting to a WiFi network
Serial.println();
Serial.print("Connecting to ");
Serial.println(ssid);
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("");
Serial.println("WiFi connected");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());
}
void callback(char* topic, byte* message, unsigned int length) {
Serial.print("Message arrived on topic: ");
Serial.print(topic);
Serial.print(". Message: ");
String messageTemp;
for (int i = 0; i < length; i++) {
Serial.print((char)message[i]);
messageTemp += (char)message[i];
}
Serial.println();
// Feel free to add more if statements to control more GPIOs with MQTT
// If a message is received on the topic esp32/output, you check if the message is either "on" or "off".
// Changes the output state according to the message
if (String(topic) == "esp32/output") {
Serial.print("Changing output to ");
if(messageTemp == "on"){
Serial.println("on");
digitalWrite(ledPin, HIGH);
}
else if(messageTemp == "off"){
Serial.println("off");
digitalWrite(ledPin, LOW);
}
}
}
void reconnect() {
// Loop until we're reconnected
while (!client.connected()) {
Serial.print("Attempting MQTT connection...");
// Attempt to connect
if (client.connect("ESP32Client", mqtt_user, mqtt_password)) {
Serial.println("connected");
// Subscribe
client.subscribe("esp32/output");
} else {
Serial.print("failed, rc=");
Serial.print(client.state());
Serial.println(" try again in 5 seconds");
// Wait 5 seconds before retrying
delay(20000);
}
}
}
void loop() {
if (!client.connected()) {
reconnect();
}
client.loop();
long now = millis();
if (now - lastMsg > 5000) {
lastMsg = now;
// Temperature in Celsius
temperature = bme.readTemperature();
// Uncomment the next line to set temperature in Fahrenheit
// (and comment the previous temperature line)
//temperature = 1.8 * bme.readTemperature() + 32; // Temperature in Fahrenheit
// Convert the value to a char array
char tempString[8];
dtostrf(temperature, 1, 2, tempString);
//Serial.print("Temperature: ");
//Serial.println(tempString);
//client.publish("esp32/temperature", tempString);
humidity = bme.readHumidity();
// Convert the value to a char array
char humString[8];
dtostrf(humidity, 1, 2, humString);
//Serial.print("Humidity: ");
//Serial.println(humString);
//client.publish("esp32/humidity", humString);
pressure = bme.readPressure();
//Convert the value to a char array
char preString[16];
dtostrf(pressure, 1, 2, preString);
//Serial.print("Pressure: ");
//Serial.println(preString);
//client.publish("esp32/pressure", preString);
gas = bme.readGas();
//Serial.print(bme.gas_resistance / 1000.0);
char gasString[8];
dtostrf(gas, 1, 2, gasString);
//Serial.println(" KOhms");
char macValue[13]; // Don't forget one byte for the terminating NULL...
uint64_t mac = ESP.getEfuseMac();
sprintf(macValue, "%012x", mac);
//Serial.print("ChipID: ");
//Serial.println(chipId);
//client.publish("esp32/chipid", chipId);
JSONVar data;
data["temperature"] = temperature;
data["humidity"] = humidity;
data["pressure"] = pressure;
data["macValue"] = macValue;
data["gas"] = gas;
//Serial.print("data.keys() = ");
//Serial.println(data.keys());
//Serial.print("Data = ");
//Serial.println(data);
String jsonString = JSON.stringify(data);
//Serial.print("JSON.stringify(data) = ");
//Serial.println(jsonString);
client.publish("esp32/data", jsonString.c_str());
}
}
everything got compiled perfectly. But when i tried to upload the code, this error came up:
Arduino: 1.8.9 (Windows 10), Board: "ESP32 Wrover Module, Huge APP
(3MB No OTA), QIO, 80MHz, 115200, Verbose"
Sketch uses 789282 bytes (25%) of program storage space. Maximum
is 3145728 bytes.
Global variables use 40652 bytes (12%) of dynamic memory, leaving
287028 bytes for local variables. Maximum is 327680 bytes.
esptool.py v2.6
Serial port COM7
Connecting.....
Chip is .....(revision 1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse,
Coding Scheme None
MAC: .....
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Warning: Could not auto-detect Flash size (FlashID=0xffffff,
SizeID=0xff), defaulting to 4MB
Compressed 8192 bytes to 47...
A fatal error occurred: Timed out waiting for packet content
A fatal error occurred: Timed out waiting for packet content
This report would have more information with
"Show verbose output during compilation"
option enabled in File -> Preferences.
I don't know why is this thing coming up because previously everything was working fine when I tried to incorporate BME280 and publish the available data on MQTT. Now, I have switched to bme680.
EDIT:
I tried to disconnect BME from ESP32, then uploaded the code, then connected BME back. The error did not show up, but still, the data is not being shown this time, rather something like this, continually:
rst:0x10 (RTCWDT_RTC_RESET),boot:0x33 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
ets_main.c 371
ets Jun 8 2016 00:22:57
It looks to me that ESP32-CAM is not in the bootloader mode when you try to upload the code.
Did you connect IO0 and GND pins and push the reset button?
You should see the message that ESP32 is ready for upload in the terminal window.
Sorry for the late reply ... :)

MySQL command to delete a row in C is not working

I don't usually code using C programming language but I learned it in school (so please bear with me because I am still a newbie).
In short, I was recently assigned to write code in C in order to delete rows from a table in MySQL database.
I used stackoverflow and other resources to help me with this code!
This is my code (not all of it):
void delete_rows(MYSQL *con)
{
char selection_query[256];
char deletion_query[256];
sprintf(selection_query, "SELECT id FROM <table> WHERE status = 'PROCESSING'\
AND started < DATE(NOW()) - INTERVAL %d DAY", expire_processing_days);
if (mysql_query(con, selection_query))
{
finish_with_error(con);
}
MYSQL_RES *result = mysql_store_result(con);
if (result == NULL)
{
finish_with_error(con);
}
int num_fields = mysql_num_fields(result);
MYSQL_ROW row;
while ((row = mysql_fetch_row(result)))
{
for(int i = 0; i < num_fields; i++)
{
printf("Deleting process with id: %s ", row[i] ? row[i] : "NULL");
sprintf(deletion_query, "DELETE FROM <table> WHERE id = %d", row[i]);
if (mysql_query(con, deletion_query))
{
finish_with_error(con);
}
mysql_commit(con);
}
printf("\n");
}
mysql_free_result(result);
}
int main()
{
MYSQL *con;
DB_CONN_PARAMS *params = calloc(1,sizeof(DB_CONN_PARAMS));
//just an alternative way of passing connection params, find a struct easier
strcpy(params->host, <host>);
strcpy(params->user, <user>);
strcpy(params->pass, <password>);
strcpy(params->db, <database>);
MYSQL * connect_db(DB_CONN_PARAMS *params);
con = connect_db(params);
//we don't need the struct anymore
free(params);
params = NULL;
//kill processes that are incomplete/hanging
delete_rows(con);
//close mysql connection
mysql_close(con);
return EXIT_SUCCESS;
}
So, the code above compiles and runs without any errors, it prints out the ids of the rows that I want to delete. But when I go to the database to check the rows, they are still there!
Is there anything I am missing?
Ok, I have figured it out finally!
I changed the %d to %s in the following line:
sprintf(deletion_query, "DELETE FROM WHERE id = %d", row[i]);.
Because row[i] is a string, I was blind to that.
I was able to figure it out by printing the whole MySQL command and noticed that the id passed is wrong!
Thank you everyone for your attempts to help me.

MQTT esp8266 client.subscribe() not working

I am working in a Iot based project in which want to store my data in to data base as well as use mqtt for communication between client and esp8266 . I tried to implement both mysql and mqtt in esp8266 node mcu. In a loop i first check if mqtt message is arrived and then update the database with sensor value.
Client.publish() works but Client.suscribe() doesnot work when update of databse is done.But when only mqtt is done it works fine.
#include <ESP8266WiFi.h>
#include <WiFiClient.h>
#include <MySQL_Connection.h>
#include <MySQL_Cursor.h>
#include <PubSubClient.h>
IPAddress server_addr(***, , ,); // IP of the MySQL server
char user[] = "root"; // MySQL user login username
char password[] = ""; // MySQL user login password
char ssid[] = "***"; // your SSID
char pass[] = "*****"; // your SSID Password
const char mqtt_server = "192.168.0.109";
long lastMsg = 0;
char msg[50];
int value = 0;
WiFiClient espClient;
MySQL_Connection conn((Client *)&espClient);
PubSubClient client(espClient);
void setup() {
Serial.begin(115200);
WiFi.begin(ssid, pass); // initializing the WIFI library
while ( WiFi.status() != WL_CONNECTED ) { // while loop to write dots during connecting
delay ( 500 );
Serial.print ( "." );
}
// print out information about the WIFI connection
Serial.println ( "" );
Serial.print ( "Connected to " );
Serial.println ( ssid );
Serial.print ( "IP address: " );
Serial.println ( WiFi.localIP() );
client.setServer(mqtt_server, 1883);
client.setCallback(callback);
}
void callback(char* topic, byte* payload, unsigned int length) {
Serial.print("Message arrived [");
Serial.print(topic);
Serial.print("] ");
for (int i = 0; i < length; i++) {
Serial.print((char)payload[i]);
}
Serial.println();
}
void reconnect() {
// Loop until we're reconnected
while (!client.connected()) {
Serial.print("Attempting MQTT connection...");
// Attempt to connect
if (client.connect("ESP8266Client")) {
Serial.println("connected");
// Once connected, publish an announcement...
client.publish("outTopic", "SAAIL");
// ... and resubscribe
client.subscribe("say");
} else {
Serial.print("failed, rc=");
Serial.print(client.state());
Serial.println(" try again in 5 seconds");
// Wait 5 seconds before retrying
delay(5000);
}
}
}
void loop() {
if (!client.connected()) {
reconnect();
}
client.loop();
delay(1000);
long now = millis();
if (now - lastMsg > 2000) {
lastMsg = now;
int newTemp = sht1x.readTemperatureC();
int newHum = sht1x.readHumidity();
Serial.print("temp:");
Serial.print(newTemp);
char INSERT_SQL[] = "INSERT INTO test.users (humidity,temp) VALUES (%d, %d );";
char query[255];
sprintf(query, INSERT_SQL, newHum, newTemp);
Serial.println("Recording data.");
conn.connect(server_addr, 3306, user, password);
// Initiate the query class instance
MySQL_Cursor *cur_mem = new MySQL_Cursor(&conn);
// Execute the query
cur_mem->execute(query);
// Note: since there are no results, we do not need to read any data
// Deleting the cursor also frees up memory used
delete cur_mem;
conn.close();
}
}
Actually, your Code is kept on moving inside a loop only and you are calling client. subscribe() inside reconnect() function. SO if your ESP8266 gets connected to MQTT Broker in a single hit then your Reconnect() will not be called.
Here is the code
#include <ESP8266WiFi.h>
#include <WiFiClient.h>
#include <MySQL_Connection.h>
#include <MySQL_Cursor.h>
#include <PubSubClient.h>
IPAddress server_addr(***, , ,); // IP of the MySQL server
char user[] = "root"; // MySQL user login username
char password[] = ""; // MySQL user login password
char ssid[] = "***"; // your SSID
char pass[] = "*****"; // your SSID Password
const char mqtt_server = "192.168.0.109";
long lastMsg = 0;
char msg[50];
int value = 0;
WiFiClient espClient;
MySQL_Connection conn((Client *)&espClient);
PubSubClient client(espClient);
void setup() {
Serial.begin(115200);
WiFi.begin(ssid, pass); // initializing the WIFI library
while ( WiFi.status() != WL_CONNECTED ) { // while loop to write dots during connecting
delay ( 500 );
Serial.print ( "." );
}
// print out information about the WIFI connection
Serial.println ( "" );
Serial.print ( "Connected to " );
Serial.println ( ssid );
Serial.print ( "IP address: " );
Serial.println ( WiFi.localIP() );
client.setServer(mqtt_server, 1883);
client.setCallback(callback);
connectmqtt();
}
void callback(char* topic, byte* payload, unsigned int length) {
Serial.print("Message arrived [");
Serial.print(topic);
Serial.print("] ");
for (int i = 0; i < length; i++) {
Serial.print((char)payload[i]);
}
Serial.println();
}
void reconnect() {
// Loop until we're reconnected
while (!client.connected()) {
Serial.print("Attempting MQTT connection...");
// Attempt to connect
if (client.connect("ESP8266Client")) {
Serial.println("connected");
// Once connected, publish an announcement...
client.publish("outTopic", "SAAIL");
// ... and resubscribe
client.subscribe("say");
} else {
Serial.print("failed, rc=");
Serial.print(client.state());
Serial.println(" try again in 5 seconds");
// Wait 5 seconds before retrying
delay(5000);
}
}
}
void loop() {
if (!client.connected()) {
reconnect();
}
client.loop();
delay(1000);
long now = millis();
if (now - lastMsg > 2000) {
lastMsg = now;
int newTemp = sht1x.readTemperatureC();
int newHum = sht1x.readHumidity();
Serial.print("temp:");
Serial.print(newTemp);
char INSERT_SQL[] = "INSERT INTO test.users (humidity,temp) VALUES (%d, %d );";
char query[255];
sprintf(query, INSERT_SQL, newHum, newTemp);
Serial.println("Recording data.");
conn.connect(server_addr, 3306, user, password);
// Initiate the query class instance
MySQL_Cursor *cur_mem = new MySQL_Cursor(&conn);
// Execute the query
cur_mem->execute(query);
// Note: since there are no results, we do not need to read any data
// Deleting the cursor also frees up memory used
delete cur_mem;
conn.close();
}
}
void connectmqtt()
{
client.connect("ESP8266Client");
{
Serial.println("connected");
// Once connected, publish an announcement...
// ... and resubscribe
client.subscribe("say");
client.publish("outTopic", "SAAIL");
if (!client.connected())
{
reconnect();
}
}
}

C forking stops after 5 children

I am working on a C program which makes a connection to a database, grabs a list of devices, then forks the requests and creates an SSH connection to that device. The issue I am having is that, the query, which has 700 results, is always starting from the beginning after it hits 5 forks.
Essentially, I've looked into pthread and glibc to handle threading, but some of the examples I found did not work as desired, or added too much complexity.
The problem I am having is, it will stop at 5 children, and stop, instead of finishing the rest of the 700 devices.
Example code:
#include <libssh2.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <sys/select.h>
#include <unistd.h>
#ifdef HAVE_ARPA_INET_H
#include <arpa/inet.h>
#endif
#include <sys/time.h>
#include <sys/wait.h>
#include <sys/types.h>
#include <stdlib.h>
#include <fcntl.h>
#include <errno.h>
#include <stdio.h>
#include <ctype.h>
#include <limits.h>
#include <mysql.h>
/********************************
*
* gcc -o confmgr cfgmgr.c -Wall -lpthread -lz -lm -lrt -ldl -lssh2 $(mysql_config --cflags) $(mysql_config --libs) -std=gnu99
*
********************************/
static int waitsocket(int socket_fd, LIBSSH2_SESSION *session)
{
struct timeval timeout;
int rc;
fd_set fd;
fd_set *writefd = NULL;
fd_set *readfd = NULL;
int dir;
timeout.tv_sec = 10;
timeout.tv_usec = 0;
FD_ZERO(&fd);
FD_SET(socket_fd, &fd);
/* now make sure we wait in the correct direction */
dir = libssh2_session_block_directions(session);
if(dir & LIBSSH2_SESSION_BLOCK_INBOUND)
readfd = &fd;
if(dir & LIBSSH2_SESSION_BLOCK_OUTBOUND)
writefd = &fd;
rc = select(socket_fd + 1, readfd, writefd, NULL, &timeout);
return rc;
}
int *connect_to_device(MYSQL_RES** args){
printf("%s", args[2]);
const char *hostname = "1.1.1.1";
const char *commandline = "command_to_run ";
const char *username = "static_user";
const char *password = "static_pass";
unsigned long hostaddr;
int sock;
struct sockaddr_in sin;
const char *fingerprint;
LIBSSH2_SESSION *session;
LIBSSH2_CHANNEL *channel;
int rc;
int exitcode;
char *exitsignal=(char *)"none";
int bytecount = 0;
size_t len;
LIBSSH2_KNOWNHOSTS *nh;
int type;
rc = libssh2_init (0);
if (rc != 0) {
fprintf (stderr, "libssh2 initialization failed (%d)\n", rc);
return 1;
}
hostaddr = inet_addr(hostname);
/* Ultra basic "connect to port 22 on localhost"
* Your code is responsible for creating the socket establishing the
* connection
*/
sock = socket(AF_INET, SOCK_STREAM, 0);
sin.sin_family = AF_INET;
sin.sin_port = htons(22);
sin.sin_addr.s_addr = hostaddr;
if (connect(sock, (struct sockaddr*)(&sin),
sizeof(struct sockaddr_in)) != 0) {
fprintf(stderr, "failed to connect!\n");
return -1;
}
/* Create a session instance */
session = libssh2_session_init();
if (!session)
return -1;
/* tell libssh2 we want it all done non-blocking */
libssh2_session_set_blocking(session, 0);
/* ... start it up. This will trade welcome banners, exchange keys,
* and setup crypto, compression, and MAC layers
*/
while ((rc = libssh2_session_handshake(session, sock)) ==
LIBSSH2_ERROR_EAGAIN);
if (rc) {
fprintf(stderr, "Failure establishing SSH session: %d\n", rc);
return -1;
}
/* We could authenticate via password */
while ((rc = libssh2_userauth_password(session, username, password)) == LIBSSH2_ERROR_EAGAIN);
if (rc) {
fprintf(stderr, "Authentication by password failed.\n");
goto shutdown;
}
libssh2_trace(session, LIBSSH2_TRACE_TRANS | LIBSSH2_TRACE_KEX | LIBSSH2_TRACE_AUTH | LIBSSH2_TRACE_CONN | LIBSSH2_TRACE_SCP | LIBSSH2_TRACE_SFTP | LIBSSH2_TRACE_ERROR | LIBSSH2_TRACE_PUBLICKEY );
/* Exec non-blocking on the remove host */
while( (channel = libssh2_channel_open_session(session)) == NULL &&
libssh2_session_last_error(session,NULL,NULL,0) == LIBSSH2_ERROR_EAGAIN )
{
waitsocket(sock, session);
}
if( channel == NULL )
{
fprintf(stderr,"Error\n");
exit( 1 );
}
while( (rc = libssh2_channel_exec(channel, commandline)) == LIBSSH2_ERROR_EAGAIN )
{
waitsocket(sock, session);
}
if( rc != 0 )
{
fprintf(stderr,"Error\n");
exit( 1 );
}
for( ;; )
{
// loop until we block
int rc;
do
{
char buffer[0x4000];
/* strange thing */
sleep( 1 );
rc = libssh2_channel_read( channel, buffer, sizeof(buffer) );
if( rc > 0 )
{
int i;
for( i=0; i < rc; ++i )
putchar( buffer[i] );
}
}
while( rc > 0 );
// this is due to blocking that would occur otherwise so we loop on this condition
if( rc == LIBSSH2_ERROR_EAGAIN )
{
waitsocket(sock, session);
}
else if( rc == 0 )
break;
}
while( (rc = libssh2_channel_close(channel)) == LIBSSH2_ERROR_EAGAIN )
;
if( rc == 0 )
{
//does-not-work if( libssh2_channel_wait_closed(channel) == 0 )
exitcode = libssh2_channel_get_exit_status( channel );
}
printf("\n%d\n", 221 );
libssh2_channel_free(channel);
channel = NULL;
/***********************/
shutdown:
libssh2_session_disconnect(session, "Normal Shutdown, Thank you for playing");
libssh2_session_free(session);
close(sock);
fprintf(stderr, "\n----------------------\nScript Finished\n\n");
libssh2_exit();
return 7;
}
/********************************
*
*
*
*
********************************/
int main(int argc, char *argv[]){
pid_t childPID;
int children = 0;
MYSQL *conn;
MYSQL_RES *res;
MYSQL_ROW row;
char *mySQLserver = "localhost";
char *mySQLuser = "root";
char *mySQLpassword = ""; /* set me first */
char *mySQLdatabase = "Devices";
conn = mysql_init(NULL);
/* Connect to database */
if (!mysql_real_connect(conn, mySQLserver,
mySQLuser, mySQLpassword, mySQLdatabase, 0, NULL, 0)) {
fprintf(stderr, "%s\n", mysql_error(conn));
exit(1);
}
/* send SQL query */
if (mysql_query(conn, "SELECT Hostname,Descr,IP,Username,Password FROM All_Active_Devices")) {
fprintf(stderr, "%s\n", mysql_error(conn));
exit(1);
}
res = mysql_use_result(conn);
/* output table name */
printf("MySQL Tables in mysql database:\n");
while ((row = mysql_fetch_row(res)) != NULL){
printf("%s \n", row[0]);
children++; // Last fork() was successful
while (children >= 5)
{
int status;
// Wait for one child to exit
if (wait(&status) == 7)
{
children--;
}
}
childPID = fork ();
if (childPID < 0) {
printf("Fork Error \n");
} else if (childPID == 0) {
printf("\tCreating Fork for %s: pid %d \n", row[0], childPID);
connect_to_device ( &row );
}
else{
printf("\tDid not create Fork for %s \n", row[0]);
}
}
/* close connection */
mysql_free_result(res);
mysql_close(conn);
return 0;
}
Your child processes are exitting - in particular not with exit status 7 - you have a return in your connect_to_device function, but that is ignored, and each child starts going round the loop, creating more children.
You probably want: return connect_to_device ( &row ); instead.
wait() returns the child PID that died, not its status - which is in WEXITSTATUS(status).
What happens after connect_to_device returns? It looks like that thread will also start in on the while loop as I don't see that child process exiting, sitting you in the while children >=5 loop. The 5 in your condition and 5 threads isn't a coincidence.
Some output from a run would be helpful, also paring down the code.
Try to get your scaffolding working, without the ssh code. Just getting the processes to stop and start shouldn't need ssh. Then add in the application logic once you're sure the underlying support works.
Perhaps ServerAliveCountMax is getting in the way:
EDIT FROM NETCODER BELOW
ServerAliveCountMax
Sets the number of server alive messages (see
below) which may be sent without ssh(1) receiving any messages back
from the server. If this threshold is reached while server alive
messages are being sent, ssh will disconnect from the server,
terminating the session. It is important to note that the use of
server alive messages is very different from TCPKeepAlive (below). The
server alive messages are sent through the encrypted channel and
therefore will not be spoofable. The TCP keepalive option enabled by
TCPKeepAlive is spoofable. The server alive mechanism is valuable when
the client or server depend on knowing when a connection has become
inactive.
The default value is 3. If, for example, ServerAliveInterval (see
below) is set to 15 and ServerAliveCountMax is left at the default, if
the server becomes unresponsive, ssh will disconnect after
approximately 45 seconds. This option applies to protocol version 2
only.
See the man page for ssh_config (man 5 ssh_config) for details.