I am trying to get the SQLite JSON functions to work with Go and the github.com/mattn/go-sqlite3 driver but not having much luck.
Using VSCode as my editor I did:
go get github.com/mattn/go-sqlite3
go build --tags "json1" github.com/mattn/go-sqlite3
Wrote my test app which calls the following function
func IsJSONValid(payloadID int) bool {
var isValid int
sql := `select json_valid(json_payload) from exec_payload where payload_id = ?`
err = database.QueryRow(sql, payloadID).Scan(&isValid)
if err != nil {
log.Println(err)
log.Println("Fatal - unable to validate JSON payload")
LogDb("IsJSONValid", "unable to validate JSON payload")
return false
}
if isValid == 0 {
return false
}
return true
}
Executing the code/func I get the following error:
no such function: json_valid
I know I can write a Struct and handle the logic to validate the json data stored in the sqlite table column - however it's pretty convenient to use / have the function already available in the DB - running the same query in the sqlite command line tool executes without any issues.
Related
I'm working on a translation project on my Angular app. I already create all the different keys for that. I try now to use Go Programming Language to add some functionalities in my translation, to work quickly after.
I try to code a function in Go Programming Language in order to read an input user on the command line. I need to read this input file in order to know if there is missing key inside. This input user must be a JSON file. I have a problem with this function, is blocked at functions.Check(err), in order to debug my function I displayed the different variable with fmt.Printf(variable to display).
I call this function readInput() in my main function.
The readInput() function is the following :
// this function is used to read the user's input on the command line
func readInput() string {
// we create a reader
reader := bufio.NewReader(os.Stdin)
// we read the user's input
answer, err := reader.ReadString('\n')
// we check if any errors have occured while reading
functions.Check(err)
// we trim the "\n" from the answer to only keep the string input by the user
answer = strings.Trim(answer, "\n")
return answer
}
In my main function I call readInput() for a specific command I created. This command line is usefull to update a JSON file and add a missing key automatically.
My func main is :
func main() {
if os.Args[1] == "update-json-from-json" {
fmt.Printf("please enter the name of the json file that will be used to
update the json file:")
jsonFile := readInput()
fmt.Printf("please enter the ISO code of the locale for which you want to update the json file: ")
// we read the user's input
locale := readInput()
// we launch the script
scripts.AddMissingKeysToJsonFromJson(jsonFile, locale)
}
I can give you the command line I use for this code go run mis-t.go update-json-from-json
Do you what I'm missing in my code please ?
Presuming that the file contains dynamic and unknown keys and values, and you cannot model them in your application. Then you can do something like:
func main() {
if os.Args[1] == "update-json-from-json" {
...
jsonFile := readInput()
var jsonKeys interface{}
err := json.Unmarshal(jsonFile, &jsonKeys)
functions.Check(err)
...
}
}
to load the contents into the empty interface, and then use the go reflection library (https://golang.org/pkg/reflect/) to iterate over the fields, find their names and values and update them according to your needs.
The alternative is to Unmarshal into a map[string]string, but that won't cope very well with nested JSON, whereas this might (but I haven't tested it).
I have been trying to setup a JSON config file to setup dynamic routes for my application. The idea is that I will be able to setup my own URL structure depending on who is using the service. I have a struct that takes in JSON and that works fine. I am using gorilla mux.
type CustomRoute struct {
Name string
Method string
Path string
HandleFunc string
}
The JSON is basically identical to the struct and it goes in fine.
The issue I have is getting the HandleFunc section.
Here is the code:
func NewRouter() *mux.Router {
routerInstance := mux.NewRouter().StrictSlash(true)
/*
All routes from the routing table
*/
// r = []CustomRoute with the JSON data
r := loadRoute()
for _, route := range r {
var handler http.Handler
handler = route.HandlerFunc
handler = core.Logger(handler, route.Name)
routerInstance.
Methods(route.Method).
Path(route.Path).
Name(route.Name).
Handler(handler)
}
return routerInstance
}
I always get the following error (as one would expect)
cannot use route.HandlerFunc (type string) as type http.Handler in assignment:
string does not implement http.Handler (missing ServeHTTP method)
I was told to use something like:
var functions = map[string]interface{}{
"HandleFunc1": HandleFunc1,
}
But I have no idea how to make this work
Thanks to RayenWindspear I was able to fix the problem. It was very simple (like everything). The map code should look like this:
var functions = map[string]http.HandlerFunc{
"HandleFunc1": HandleFunc1,
}
I am using a mux for subdomains, so my examples may be a bit off. The map you are being told to use is done something like this:
type Handlers map[string]http.HandlerFunc
func (handlers Handlers) ServeHTTP(w http.ResponseWriter, r *http.Request) {
path := r.URL.Path
if handle := handlers[path]; handle != nil {
handle.ServeHTTP(w, r)
} else {
http.Error(w, "Not found", 404)
}
}
I'm trying to move old LUA method which was loading some JSON content from file into global variable into "class". But I get following errors all the time:
attempt to call field 'decode' (a nil value)
attempt to index global 'cjson' (a nil value)
I don't know lua well but i tried almost all combinations without result so can you explain why this errors occurs?
Current implementation of module looks like:
Config = {}
Config.__index = Config
function Config.create(config_filename)
local cjson = require("cjson")
local config = {}
setmetatable(config,Config)
local f = io.open(config_filename, "r")
local content = f:read("*a")
f:close()
config = cjson.decode(content)
return config
end
return Config
As final result I want to execute something like this from other file:
local config_class = require("config")
local config = config_class.create("/path/to/file.json")
ngx.say(config:some_configuration_data())
As the error message tells you cjson and decode are nil values which cannot be indexed or called.
require will load some file and run the contained code and pass the return value through. If you run a Lua script it behaves like a function which returns nil by default. So unless you specify what the script returns require will return nil.
I don't know what is inside your cjson file that you require but it obviously does not return the wanted json implementation but nil.
So the code in cjson should return a Lua table with a function stored under key "decode".
I'm new in golang development and have some question regarding something related to this question.
As a learning exercise, I'm trying to create a simple library to handle json based configuration file. As a configuration file to be used for more then one app, it should be able to handle different parameters. Then I have created a type struct Configuration that has the filename and a data interface. Each app will have a struct based on its configuration needs.
In the code bellow, I put all together (lib and "main code") and the "TestData struct" is the "app parameters".
If it doesn't exists, it will set a default values and create the file, and it is working. But when I try to read the file. I try to decode the json and put it back into the data interface. But it is giving me an error and I couldn't figure out how to solve this. Can someone help on this?
[updated] I didn't put the targeted code before, because I though that it would be easier to read in in all as a single program. Bellow is the 'targeted code' for better view of the issue.
As I will not be able to use the TestData struct inside the library, since it will change from program to program, the only way to handle this was using interface. Is there a better way?
library config
package config
import (
"encoding/json"
"fmt"
"os"
)
// Base configuration struct
type Configuration struct {
Filename string
Data interface{}
}
func (c *Configuration) Create(cData *Configuration) bool {
cFile, err := os.Open(cData.Filename)
defer cFile.Close()
if err == nil {
fmt.Println("Error(1) trying to create a configuration file. File '", cData.Filename, "' may already exist...")
return false
}
cFile, err = os.Create(cData.Filename)
if err != nil {
fmt.Println("Error(2) trying to create a configuration file. File '", cData.Filename, "' may already exist...")
return false
}
buffer, _ := json.MarshalIndent(cData.Data, "", "")
cFile.Write(buffer)
return true
}
func (c *Configuration) Read(cData *Configuration) bool {
cFile, err := os.Open(cData.Filename)
defer cFile.Close()
if err != nil {
fmt.Println("Error(1) trying to read a configuration file. File '", cData.Filename, "' may not already exist...")
return false
}
jConfig := json.NewDecoder(cFile)
jerr := jConfig.Decode(&cData.Data)
if jerr != nil {
panic(jerr)
}
return true
}
program using library config
package main
import (
"fmt"
"./config"
)
// struct basic para configuração
type TestData struct {
URL string
Port string
}
func main() {
var Config config.Configuration
Config.Filename = "config.json"
if !Config.Read(&Config) {
Config.Data = TestData{"http", "8080"}
Config.Create(&Config)
}
fmt.Println(Config.Data)
TestData1 := &TestData{}
TestData1 = Config.Data.(*TestData) // error, why?
fmt.Println(TestData1.URL)
}
NEW UPDATE:
I have made some changes after JimB comment about I'm not clear about some concepts and I tried to review it. Sure many things aren't clear for me yet unfortunately. The "big" understanding I believe I got, but what mess my mind up is the "ins" and "outs" of values and formats and pointers, mainly when it goes to other libraries. I'm not able yet to follow the "full path" of it.
Yet, I believe I had some improvement on my code.
I think that I have corrected some points, but still have some big questions:
I stopped sending "Configuration" as a parameter as all "data" were already there as they are "thenselfs" in the instance. Right?
Why do I have use reference in the line 58 (Config.Data = &TestData{})
Why to I have to use pointer in the line 64 (tmp := Config.Data.(*TestData)
Why I CANNOT use reference in line 69 (Config.Data = tmp)
Thanks
The reason you are running into an error is because you are trying to decode into an interface{} type. When dealing with JSON objects, they are decoded by the encoding/json package into map[string]interface{} types by default. This is causing the type assertion to fail since the memory structure for a map[string]interface{} is much different than that of a struct.
The better way to do this is to make your TestData struct the expected data format for your Configuration struct:
// Base configuration struct
type Configuration struct {
Filename string
Data *TestData
}
Then when Decoding the file data, the package will unmarshal the data into the fields that match the closest with the data it finds.
If you need more control over the data unmarshaling process, you can dictate which JSON fields get decoded into which struct members by using struct tags. You can read more about the json struct tags available here: https://golang.org/pkg/encoding/json/#Marshal
You are trying to assert that Config.Data is of type *TestData, but you're assigning it to TestData{"http", "8080"} above. You can take the address of a composite literal to create a pointer:
Config.Data = &TestData{"http", "8080"}
If your config already exsits, your Read method is going to fill in the Data field with the a default json data type, probably a map[string]interface{}. If you assign a pointer of the correct type to Data first, it will decode into the expected type.
Config.Data = &TestData{}
Ans since Data is an interface{}, you do not want to ever use a pointer to that value, so don't use the & operator when marshaling and unmarshaling.
I need to transfer a MongoDB query to a different system. For this reason I would like to use the MongoDB Extended JSON. I need this to be done mostly because I use date comparisons in my queries.
So, the kernel of the problem is that I need to transfer a MongoDB query that has been generated in a node.js back-end to another back-end written in Go language.
Intuitively, the most obvious format for sending this query via REST, is JSON. But, MongoDB queries are not exactly JSON, but BSON, which contains special constructs for dates.
So, the idea is to convert the queries into JSON using MongoDB Extended JSON as form of representation of the special constructs. After some tests it's clear that these queries do not work. Both the MongoDB shell and queries sent via node.js's need the special ISODate or new Date constructs.
Finally, the actual question: are there functions to encode/decode from JSON to BSON, taking into account MongoDB Extended JSON, both in JavaScript (node.js) and Go language?
Updates
Node.js encoding package
Apparently there is a node.js package that parses and stringifies BSON/JSON.
So, half of my problem is resolved. I wonder if there is something like this in Go language.
Sample query
For example, the following query is in normal BSON:
{ Tmin: { $gt: ISODate("2006-01-01T23:00:00.000Z") } }
Translated into MongoDB Extended JSON, it becomes:
{ "Tmin": { "$gt" : { "$date" : 1136156400000 }}}
After some research I found the mejson library, however it's for Marshaling only, so I decided to write an Unmarshaller.
Behold ejson (I wrote it), right now it's a very simple ejson -> bson converter, there's no bson -> ejson yet, you can use mejson for that.
An example:
const j = `{"_id":{"$oid":"53c2ab5e4291b17b666d742a"},"last_seen_at":{"$date":1405266782008},"display_name":{"$undefined":true},
"ref":{"$ref":"col2", "$id":"53c2ab5e4291b17b666d742b"}}`
type TestS struct {
Id bson.ObjectId `bson:"_id"`
LastSeenAt *time.Time `bson:"last_seen_at"`
DisplayName *string `bson:"display_name,omitempty"`
Ref mgo.DBRef `bson:"ref"`
}
func main() {
var ts TestS
if err := ejson.Unmarshal([]byte(j), &ts); err != nil {
panic(err)
}
fmt.Printf("%+v\n", ts)
//or to convert the ejson to bson.M
var m map[string]interface{}
if err := json.Unmarshal([]byte(j), &m); err != nil {
t.Fatal(err)
}
err := ejson.Normalize(m)
if err != nil {
panic(err)
}
fmt.Printf("%+v\n", m)
}