I have installed FIWARE on my machine (Ubuntu 18.04) and I am currently trying to work with the IoT Agent, using the HTTPBindings.js (my data is sent via LoRaWAN and I've changed the parseData function in order to use my own data "protocol" [id=1&temp=12&humidity=10], which brings me here to ask 2 questions for someone who is more experienced and can help me with:
function parseData(req, res, next) {
let data;
let error;
let payload;
let obj;
try {
let newPayload = new Buffer.from(payload, "base64").toString("ascii");
var ps = newPayload.split("&").reduce((accum, x) => {
const kv = x.split("=");
return { ...accum, ...{ [kv[0]]: kv[1] } };
}, {});
data = ulParser.parse(newPayload.replace(/&/g, "|").replace(/=/g, "|"));
} catch (e) {
error = e;
}
if (error) {
next(error);
} else {
req.ulPayload = data;
config.getLogger().debug(context, 'Parsed data: [%j]', data);
next();
}
}
After changing this function, I can't get the data to be updated in the orion/v2/entities.. Would someone explain me how does this work?
How can I add a proxy to usenter code heree in the Wirecloud? I've created it using the FIWARE servers, but testing on my own, I do not have this.
Thank you in advance.
Configuring NGSI Proxy
The ngsi-proxy is configured using environment variables and a port.
ngsi-proxy:
image: fiware/ngsiproxy:1.2.0
hostname: ngsi-proxy
container_name: wc-ngsi-proxy
networks:
default:
ipv4_address: 172.18.1.14
expose:
- "8100"
ports:
- "8100:8100"
environment:
- PORT=8100
- TRUST_PROXY_HEADERS=0
The NGSI proxy config in the wirecloud widget is then http://<host>:<port> - in this case http://ngsi-proxy:8100
Testing HTTP Binding connectivity
Incoming HTTP measures can be controlled by the IOTA_HTTP_PORT Environment variable:
iot-agent:
image: fiware/iotagent-ul:${ULTRALIGHT_VERSION}
hostname: iot-agent
container_name: fiware-iot-agent
depends_on:
- mongo-db
- orion
networks:
- default
ports:
- "4041:4041"
- "7896:7896"
expose:
- "7896"
environment:
..etc
- IOTA_NORTH_PORT=4041
- IOTA_LOG_LEVEL=DEBUG
- IOTA_HTTP_PORT=7896
- IOTA_PROVIDER_URL=http://iot-agent:4041
If you ramp up the debug and expose the relevant port, you should be able to send measures directly to your custom IoT Agent and see some sort of response (probably an error) - this can help track down your coding issue.
You can always add additional debug logging to the custom IoT Agent to see how the conversion is working.
Related
I have a problem now. I have tried a few methods from Google to control services start sequence.for example, use wait-for-it.sh、 write shell script ,but all doesn't work for me.
I'm using macOS and my project is asp.net core web api in .net 6 core and using entity framework to connect and migrate to mysql database.
dockerfile
`# Get base sdk Image from Microsoft
FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build-env
WORKDIR /app
#copy the csprog file and restore any dependecies
COPY *.csproj ./
RUN dotnet restore
#copy the project files and build our release
COPY . ./
RUN dotnet publish -c Release -o out
#Generate runtime image
FROM mcr.microsoft.com/dotnet/aspnet:6.0
WORKDIR /app
EXPOSE 80
COPY --from=build-env /app/out .
ENTRYPOINT ["dotnet", "SuperHeroAPI.dll"]`
docker-compose.yml
`version: '3.4'
services:
api:
build:
context: .
dockerfile: Dockerfile
ports:
- "8080:80"
container_name: super
depends_on:
- db
restart: always
environment:
- DBHOST=db
- ASPNETCORE_ENVIRONMENT=Development
db:
image: mysql:8.0.29
container_name: db
environment:
MYSQL_ROOT_PASSWORD: 123456
MYSQL_DATABASE: apidb
ports:
- "3306:3306"
`
program.cs
`
var builder = WebApplication.CreateBuilder(args);
// Add services to the container.
builder.Services.AddControllers();
builder.Services.AddCors();
var host = builder.Configuration["DBHOST"] ?? "db";
var port = builder.Configuration["DBPORT"] ?? "3306";
var password = builder.Configuration["DBPASSWORD"] ?? "123456";
builder.Services.AddDbContext<DataContext>(options =>
{
options.UseMySql($"server={host};userid=root;pwd={password};port={port};database=apidb", new MySqlServerVersion("8.0.29"));
});
var dataContext = builder.Services.BuildServiceProvider().GetService<DataContext>();
dataContext.Database.Migrate();
// Learn more about configuring Swagger/OpenAPI at https://aka.ms/aspnetcore/swashbuckle
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();
builder.Services.AddScoped<ISuperHeroDao, SuperHeroImpl>();
builder.Services.AddScoped<SuperHeroService>();
var app = builder.Build();
// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
app.UseSwagger();
app.UseSwaggerUI();
}
app.UseHttpsRedirection();
app.UseAuthorization();
app.MapControllers();
app.Run();
Datacontext.cs
`using Microsoft.EntityFrameworkCore;
namespace SuperHeroAPI.Data
{
public class DataContext : DbContext
{
public DataContext(DbContextOptions<DataContext> options) : base(options) { }
public DbSet<SuperHero> SuperHeroes { get; set; }
}
}`
based on above all code , I can start my api service finally! but api would restart repeatedly until db service start successfully.I don't want that . I wanna start db service until succeed and then start api service so it won't restart repeatedly.
I'm sorry I'm not a native english speaker ,maybe I can't express my idea clearly , I would appreciate you if you could provide useful solutions and ideas.
expect to start db service first until succeed and then start api service. I've tried wait-for-it.sh. shell scripts .doesn't work.
I want to use Docker to make a container that contains a MySQL DB with Express to expose into certain ports for example 8080 so I can fetch that using React for the view.
Right now I made simple server with express that fetches the data from a MySQL database and displays it on 'localhost:8000/posts' (posts is where i am using SQL Query to select everything from the table.
This is a index.ts file:
//Main File
import express from "express";
import { connection } from "./db-connect";
const app = express();
const port = 8000;
var cors = require("cors");
/*
cors was installed so I could get the data
and display it to react, otherwise I was getting an error
in the console.
*/
app.use(cors());
app.listen(80, function() {
console.log("CORS-enabled web server listening on port 80");
});
app.get("/", (req, res) => {
res.send("Go to /posts to see the Data");
});
app.get("/posts", (req, res) => {
connection.query("SELECT * FROM blog.posts", (err, result) => {
if (err) {
return res.send(err);
} else {
return res.json(result);
}
});
});
app.listen(port, err => {
if (err) {
return console.error(err);
}
return console.log(`server is listening on ${port}`);
});
So I have a database named blog with table called posts
With this I am getting an array with data as an object which is I wanted.
What I wanted to learn is Dockerize this into a container. With MySQL that has blog database with posts table that contains id, title, body and user.
After I'd like to host the Docker that contains the db and deploy it to digitalocean or similar service.
I'd suggest to use docker-compose for this.
You'd then have a .yml file that looks something like this:
version: '3.3'
services:
db:
image: mysql:5.7
restart: always
environment:
# Use this to connect to your dockerized mysql container
MYSQL_DATABASE: 'db'
MYSQL_USER: 'user'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password'
ports:
- '3306:3306'
expose:
# Use this port in your express app
- '3306'
volumes:
- my-db:/var/lib/mysql
volumes:
my-db:
Then you need to dockerize your express app and include it in the docker-compose.yml file. Here is a guide on how to dockerize a node.js app such as yours.
Once you succeeded to connect the two containers in docker-compose, deploying it on a Digital Ocean server is just as easy as installing Docker-Compose on the server (if it's not already installed), copying your code to the server and running docker-compose up --build.
I have a project that uses NodeJS as a server (with ExpressJS) and MySQL to handle databases. To load them both together, I am using Docker. Although this project includes a ReactJS client (and I have a client folder for the react and a server folder for the nodejs), I have tested communication between the server and client and it works. Here is the code that pertains to both the server and mysql services:
docker-compose.yml
mysql:
image: mysql:5.7
environment:
MYSQL_HOST: localhost
MYSQL_DATABASE: sampledb
MYSQL_USER: gfcf14
MYSQL_PASSWORD: xxxx
MYSQL_ROOT_PASSWORD: root
ports:
- 3307:3306
restart: unless-stopped
volumes:
- /var/lib/mysql
- ./db/greendream.sql:/docker-entrypoint-initdb.d/greendream.sql
.
.
.
server:
build: ./server
depends_on:
- mysql
expose:
- 8000
environment:
API_HOST: "http://localhost:3000/"
APP_SERVER_PORT: 8000
ports:
- 8000:8000
volumes:
- ./server:/app
links:
- mysql
command: yarn start
Then there is the Dockerfile for the server:
FROM node:10-alpine
RUN mkdir -p /app
WORKDIR /app
COPY package.json /app
COPY yarn.lock /app
RUN yarn install
COPY . /app
CMD ["yarn", "start"]
In the server's package.json, the script start is simply this: "start": "nodemon index.js"
And the file index.js that gets executed is this:
const express = require('express');
const cors = require('cors');
const mysql = require('mysql');
const app = express();
const con = mysql.createConnection({
host: 'localhost',
user: 'gfcf14',
password: 'xxxx',
database: 'sampledb',
});
app.use(cors());
app.listen(8000, () => {
console.log('App server now listening on port 8000');
});
app.get('/test', (req, res) => {
con.connect(err => {
if (err) {
res.send(err);
} else {
res.send(req.query);
}
})
});
So all I want to do for now is confirm that a connection takes place. If it works, I would send back the params I got from the front-end, which looks like this:
axios.get('http://localhost:8000/test', {
params: {
test: 'hi',
},
}).then((response) => {
console.log(response.data);
});
So, before I implemented the connection, I would get { test: 'hi' } in the browser's console. I expect to get that as soon as the connection is successful, but what I get instead is this:
{
address: "127.0.0.1"
code: "ECONNREFUSED"
errno: "ECONNREFUSED"
fatal: true
port: 3306
syscall: "connect"
__proto__: Object
}
I thought that maybe I have the wrong privileges, but I also tried it using root as user and password, but I get the same. Weirdly enough, if I refresh the page I don't get an ECONNREFUSED, but a PROTOCOL_ENQUEUE_AFTER_FATAL_ERROR (with a fatal: false). Why would this happen if I am using the right credentials? Please let me know if you have spotted something I may have missed
In your mysql.createConnection method, you need to provide the mysql host. Mysql host is not localhost as mysql has its own container with its own IP. Best way to achieve this is to externalize your mysql host and allow docker-compose to resolve the mysql service name(in your case it is mysql) to its internal IP which is what we need. Basically, your nodejs server will connect to the internal IP of the mysql container.
Externalize the mysql host in nodejs server:
const con = mysql.createConnection({
host: process.env.MYSQL_HOST_IP,
...
});
Add this in your server service in docker-compose:
environment:
MYSQL_HOST_IP: mysql // the name of mysql service in your docker-compose, which will get resolved to the internal IP of the mysql container
I created a container with a mysql database and the manual connection with a .net application works very good. Now I would use a docker-compose to build the mysql container and my .net application image automatically.
My docker compose looks like that.
version: '3.1'
networks:
overlay:
services:
authentication_service:
image: authentication_service:latest
depends_on:
- "authentication_service_db"
- "adminer"
build:
context: .
dockerfile: Dockerfile
links:
- authentication_service_db
ports:
- "5003:80"
networks:
- overlay
authentication_service_db:
image: mysql:8.0.3
environment:
MYSQL_ROOT_PASSWORD: secret
MYSQL_DATABASE: authenticationdb
ports:
- 3305:3306
restart: always
volumes:
- ./data-authentication:/var/lib/mysql
networks:
- overlay
When I start the application in development the connection string is printed and looks like that server=localhost;Port=3305;Database=authenticationdb;Uid=root;Pwd=secret. This works.
When I run docker-compose up -d --build it changes:
server=authentication_service_db;Port=3305;Database=authenticationdb;Uid=root;Pwd=secret;
My connection test script:
private static void WaitForDbInit(string connectionString)
{
var connection = new MySqlConnection(connectionString);
Console.WriteLine("Connecting with credentials: {0}", connectionString);
var retries = 1;
while (retries < 5)
{
try
{
Console.WriteLine("Connecting to db. Trial: {0}", retries);
connection.Open();
connection.Close();
break;
}
catch (MySqlException)
{
Thread.Sleep((int) Math.Pow(2, retries) * 1000);
retries++;
}
}
}
The Log:
Connecting to db. Trial: 1
Connecting to db. Trial: 2
Connecting to db. Trial: 3
Connecting to db. Trial: 4
[...]
application startup exception: MySql.Data.MySqlClient.MySqlException (0x80004005): Unable to connect to any of the specified MySQL hosts.
at MySqlConnector.Core.ServerSession.ConnectAsync(ConnectionSettings cs, ILoadBalancer loadBalancer, IOBehavior ioBehavior, CancellationToken cancellationToken) in C:\projects\mysqlconnector\src\MySqlConnector\Core\ServerSession.cs:line 299
appsettings.json:
{
"Logging": {
"LogLevel": {
"Default": "Warning"
}
},
"AllowedHosts": "*",
"MysqlConnection": {
"connectionString": "server=authentication_service_db;Port=3305;Database=authenticationdb;Uid=root;Pwd=secret;"
}
}
So I don't know how debug or what I can do next. I don't get a connection when I use docker compose.
You mapped port 3306 to 3305 to your host, where you develop the application. So you can connect to db on port 3305. However, you are still using port 3305 inside the container. Because your service (authentication_service_db) is listening on port 3306, not 3305, your application having trouble to connect to database.
You must change connection string to use port 3306, instead of 3305.
I'm trying to connect to a db in a MySQL runtime from another NodeJS runtime in a multi-machine workspace.
In a test I'm calling the API http://localhost:3000/target with the list of target users. Code in this API runs a SELECT on the db:
...
exports.list = function(req, res) {
req.getConnection(function(err, connection) {
if (err) {
console.log("MySQL " + err);
} else {
connection.query('SELECT id FROM target', function(err, rows) {
if (err) {
console.log("Error Selecting : %s ", err);
} else {
...
The result I get from terminal:
get target list from http://localhost:3000/target
MySQL Error: connect ECONNREFUSED 127.0.0.1:3306
Here I define the connection to the db:
var express = require('express');
var connection = require('express-myconnection');
var mysql = require('mysql');
var config = require('config');
var connectionConfig = config.get('mysql');
var connectionInstance = connection(mysql, connectionConfig, 'request');
...
app.use(connectionInstance);
app.get('/', function(req, res) {
res.send('Welcome');
});
app.get('/target', target.list);
....
config:
{
"mysql": {
"host": "localhost",
"user": "[user]",
"password": "[password]",
"database": "[database]"
},
"app": {
"port": 3000,
"server": "http://localhost"
}
}
This is what I have in the configuration of the db machine in Eclipse Che:
snapshot of servers configuration
Here's my recipe:
services:
db:
image: eclipse/mysql
environment:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: petclinic
MYSQL_USER: petclinic
MYSQL_PASSWORD: password
MYSQL_ROOT_USER: root
mem_limit: 1073741824
dev-machine:
image: eclipse/node
mem_limit: 2147483648
depends_on:
- db
elasticsearch:
image: florentbenoit/cdvy-ela-23
mem_limit: 2147483648
Can you share your recipe for the multi-machine workspace? That would help a lot in debugging it.
Just a guess: I think the problem with your setup is the use of localhost for your db connection. If you are running a multi-machine setup, the db is running in a different docker container and needs to be addressed by its name.
Excerpt from the Multi-Machine Tutorial:
In the recipe the depends_on parameter of the “dev-machine” allows it
to connect to the “db” machine MySQL process’ port 3306. The
“dev-machine” configures its MySQL client connection in the projects
source code at src/main/resources/spring/data-access.properties. The
url is defined by jdbc.url=jdbc:mysql://db:3306/petclinic which uses
the database machine’s name “db” and the MySQL server default port
3306.
You need to configure the open ports in your recipe.
Disclaimer: I am not directly affiliated with Eclipse Che, Codenvy or Red Hat, but we are building our own cloud IDE for C/C++ multicore optimization on top of Eclipse Che.