I'm trying to parse JSON using Boost's property_tree parser and from C++11 code (my system is Debian Wheezy with gcc 4.7.2 and Boost 1.49). I tried the following code based on Serializing and deserializing json with boost:
#include <map>
#include <sstream>
#include <boost/property_tree/ptree.hpp>
#include <boost/property_tree/json_parser.hpp>
using boost::property_tree::ptree; using boost::property_tree::read_json; using boost::property_tree::write_json;
void example() {
// Write json.
ptree pt;
pt.put ("foo", "bar");
std::ostringstream buf;
write_json (buf, pt, false);
std::string json = buf.str(); // {"foo":"bar"}
// Read json.
ptree pt2;
std::istringstream is (json);
read_json (is, pt2);
std::string foo = pt2.get<std::string> ("foo");
}
If I compile this with g++ -std=c++03 -c' everything is fine. However, I also want to use C++11 features (which the code in the linked thread actually does!). But withg++ -std=c++11 -c' I get compile errors:
In file included from /usr/include/boost/property_tree/json_parser.hpp:14:0,
from test.cpp:4:
/usr/include/boost/property_tree/detail/json_parser_read.hpp: In instantiation of ‘void boost::property_tree::json_parser::context<Ptree>::a_literal_val::operator() (boost::property_tree::json_parser::context<Ptree>::It, boost::property_tree::json_parser::context<Ptree>::It) const [with Ptree = boost::property_tree::basic_ptree<std::basic_string<char>, std::basic_string<char> >; boost::property_tree::json_parser::context<Ptree>::It = __gnu_cxx::__normal_iterator<char*, std::vector<char, std::allocator<char> > >]’:
/usr/include/boost/spirit/home/classic/core/scanner/scanner.hpp:148:13: required from ‘static void boost::spirit::classic::attributed_action_policy<boost::spirit::classic::nil_t>::call(const ActorT&, boost::spirit::classic::nil_t, const IteratorT&, const IteratorT&) [with ActorT = boost::property_tree::json_parser::context<boost::property_tree::basic_ptree<std::basic_string<char>, std::basic_string<char> > >::a_literal_val; IteratorT = __gnu_cxx::__normal_iterator<char*, std::vector<char, std::allocator<char> > >]’
/usr/include/boost/spirit/home/classic/core/scanner/scanner.hpp:163:13: required from ‘void boost::spirit::classic::action_policy::do_action(const ActorT&, AttrT&, const IteratorT&, const IteratorT&) const [with ActorT = boost::property_tree::json_parser::context<boost::property_tree::basic_ptree<std::basic_string<char>, std::basic_string<char> > >::a_literal_val; AttrT = boost::spirit::classic::nil_t; IteratorT = __gnu_cxx::__normal_iterator<char*, std::vector<char, std::allocator<char> > >]’
...
test.cpp:20:1: required from here
/usr/include/boost/property_tree/detail/json_parser_read.hpp:105:17: error: no matching function for call to ‘boost::property_tree::basic_ptree<std::basic_string<char>, std::basic_string<char> >::push_back(std::pair<std::basic_string<char>, std::basic_string<char> >)’
/usr/include/boost/property_tree/detail/json_parser_read.hpp:105:17: note: candidate is:
In file included from /usr/include/boost/property_tree/ptree.hpp:516:0,
from test.cpp:3:
/usr/include/boost/property_tree/detail/ptree_implementation.hpp:362:9: note: boost::property_tree::basic_ptree<Key, Data, KeyCompare>::iterator boost::property_tree::basic_ptree<Key, Data, KeyCompare>::push_back(const value_type&) [with Key = std::basic_string<char>; Data = std::basic_string<char>; KeyCompare = std::less<std::basic_string<char> >; boost::property_tree::basic_ptree<Key, Data, KeyCompare>::value_type = std::pair<const std::basic_string<char>, boost::property_tree::basic_ptree<std::basic_string<char>, std::basic_string<char> > >]
/usr/include/boost/property_tree/detail/ptree_implementation.hpp:362:9: note: no known conversion for argument 1 from ‘std::pair<std::basic_string<char>, std::basic_string<char> >’ to ‘const value_type& {aka const std::pair<const std::basic_string<char>, boost::property_tree::basic_ptree<std::basic_string<char>, std::basic_string<char> > >&}’
How can I use Boost's read_json with C++11? Do I need a newer Boost version for this (i. e. install manually from source instead of using Wheezy's packaged one)? Is there something wrong in my code? Or is this simply not possible?
It is a known bug of older Boost versions.
You can fix it by applying the following patch:
--- json_parser_read.hpp 2013-09-01 03:55:57.000000000 +0400
+++ /usr/include/boost/property_tree/detail/json_parser_read.hpp 2013-09-01 03:56:21.000000000 +0400
## -102,7 +102,7 ##
void operator()(It b, It e) const
{
BOOST_ASSERT(c.stack.size() >= 1);
- c.stack.back()->push_back(std::make_pair(c.name, Str(b, e)));
+ c.stack.back()->push_back(std::make_pair(c.name, Ptree(Str(b, e))));
c.name.clear();
c.string.clear();
}
or with
sed -i -e 's/std::make_pair(c.name, Str(b, e))/std::make_pair(c.name, Ptree(Str(b, e)))/' json_parser_read.hpp
Related
When calling a swig generated function returning std::tuple, i get a swig object of that std::tuple.
Is there a way to use type-maps or something else to extract the values? I have tried changing the code to std::vector for a small portion of the code, and that works. (using %include <std_vector.i> and templates) But i don't want to make too many changes in the C++ part.
Edit: here is a minimal reproducible example:
foo.h
#pragma once
#include <tuple>
class foo
{
private:
double secret1;
double secret2;
public:
foo();
~foo();
std::tuple<double, double> return_thing(void);
};
foo.cpp
#include "foo.h"
#include <tuple>
foo::foo()
{
secret1 = 1;
secret2 = 2;
}
foo::~foo()
{
}
std::tuple<double, double> foo::return_thing(void) {
return {secret1, secret2};
}
foo.i
%module foo
%{
#include"foo.h"
%}
%include "foo.h"
When compiled on my linux using
-:$ swig -python -c++ -o foo_wrap.cpp foo.i
-:$ g++ -c foo.cpp foo_wrap.cpp '-I/usr/include/python3.8' '-fPIC' '-std=c++17' '-I/home/simon/Desktop/test_stack_overflow_tuple'
-:$ g++ -shared foo.o foo_wrap.o -o _foo.so
I can import it in python as shown:
test_module.ipynb
import foo as f
Foo = f.foo()
return_object = Foo.return_thing()
type(return_object)
print(return_object)
Outputs is
SwigPyObject
<Swig Object of type 'std::tuple< double,double > *' at 0x7fb5845d8420>
Hopefully this is more helpful, thank you for responding
To clarify i want to be able to use the values in python something like this:
main.cpp
#include "foo.h"
#include <iostream>
//------------------------------------------------------------------------------'
using namespace std;
int main()
{
foo Foo = foo();
auto [s1, s2] = Foo.return_thing();
cout << s1 << " " << s2 << endl;
}
//------------------------------------------------------------------------------
Github repo if anybody is interested
https://github.com/simon-cmyk/test_stack_overflow_tuple
Our goal is to make something like the following SWIG interface work intuitively:
%module test
%include "std_tuple.i"
%std_tuple(TupleDD, double, double);
%inline %{
std::tuple<double, double> func() {
return std::make_tuple(0.0, 1.0);
}
%}
We want to use this within Python in the following way:
import test
r=test.func()
print(r)
print(dir(r))
r[1]=1234
for x in r:
print(x)
i.e. indexing and iteration should just work.
By re-using some of the pre-processor tricks I used to wrap std::function (which were themselves originally from another answer here on SO) we can define a neat macro that "just wraps" std::tuple for us. Although this answer is Python specific it should in practice be fairly simple to adapt for most other languages too. I'll post my std_tuple.i file, first and then annotate/explain it after:
// [1]
%{
#include <tuple>
#include <utility>
%}
// [2]
#define make_getter(pos, type) const type& get##pos() const { return std::get<pos>(*$self); }
#define make_setter(pos, type) void set##pos(const type& val) { std::get<pos>(*$self) = val; }
#define make_ctorargN(pos, type) , type v##pos
#define make_ctorarg(first, ...) const first& v0 FOR_EACH(make_ctorargN, __VA_ARGS__)
// [3]
#define FE_0(...)
#define FE_1(action,a1) action(0,a1)
#define FE_2(action,a1,a2) action(0,a1) action(1,a2)
#define FE_3(action,a1,a2,a3) action(0,a1) action(1,a2) action(2,a3)
#define FE_4(action,a1,a2,a3,a4) action(0,a1) action(1,a2) action(2,a3) action(3,a4)
#define FE_5(action,a1,a2,a3,a4,a5) action(0,a1) action(1,a2) action(2,a3) action(3,a4) action(4,a5)
#define GET_MACRO(_1,_2,_3,_4,_5,NAME,...) NAME
%define FOR_EACH(action,...)
GET_MACRO(__VA_ARGS__, FE_5, FE_4, FE_3, FE_2, FE_1, FE_0)(action,__VA_ARGS__)
%enddef
// [4]
%define %std_tuple(Name, ...)
%rename(Name) std::tuple<__VA_ARGS__>;
namespace std {
struct tuple<__VA_ARGS__> {
// [5]
tuple(make_ctorarg(__VA_ARGS__));
%extend {
// [6]
FOR_EACH(make_getter, __VA_ARGS__)
FOR_EACH(make_setter, __VA_ARGS__)
size_t __len__() const { return std::tuple_size<std::decay_t<decltype(*$self)>>{}; }
%pythoncode %{
# [7]
def __getitem__(self, n):
if n >= len(self): raise IndexError()
return getattr(self, 'get%d' % n)()
def __setitem__(self, n, val):
if n >= len(self): raise IndexError()
getattr(self, 'set%d' % n)(val)
%}
}
};
}
%enddef
This is just the extra includes we need for our macro to work
These apply to each of the type arguments we supply to our %std_tuple macro invocation, we need to be careful with commas here to keep the syntax correct.
This is the mechanics of our FOR_EACH macro, which invokes each action per argument in our variadic macro argument list
Finally the definition of %std_tuple can begin. Essentially this is manually doing the work of %template for each specialisation of std::tuple we care to name inside of the std namespace.
We use our macro for each magic to declare a constructor with arguments for each element of the correct type. The actual implementation here is the default one from the C++ library which is exactly what we need/want though.
We use our FOR_EACH macro twice to make a member function get0, get1, getN of the correct type of each tuple element and the correct number of them for the template argument size. Likewise for setN. Doing it this way allows the usual SWIG typemaps for double, etc. or whatever types your tuple contains to be applied automatically and correctly for each call to std::get<N>. These are really just an implementation detail, not intended to be part of the public interface, but exposing them makes no real odds.
Finally we need an implementation of __getitem__ and a corresponding __setitem__. These simply look up and call the right getN/setN function on the class and call that instead. We take care to raise IndexError instead of the default exception if an invalid index is used as this will stop iteration correctly when we try to iterate of the tuple.
This is then sufficient that we can run our target code and get the following output:
$ swig3.0 -python -c++ -Wall test.i && g++ -shared -o _test.so test_wrap.cxx -I/usr/include/python3.7 -m32 && python3.7 run.py
<test.TupleDD; proxy of <Swig Object of type 'std::tuple< double,double > *' at 0xf766a260> >
['__class__', '__del__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattr__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__len__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', '__swig_destroy__', '__swig_getmethods__', '__swig_setmethods__', '__weakref__', 'get0', 'get1', 'set0', 'set1', 'this']
0.0
1234.0
Generally this should work as you'd hope in most input/output situations in Python.
There are a few improvements we could look to make:
Implement repr
Implement slicing so that tuple[n:m] type indexing works
Handle unpacking like Python tuples.
Maybe do some more automatic conversions for compatible types?
Avoid calling __len__ for every get/setitem call, either by caching the value in the class itself, or postponing it until the method lookup fails?
Suppose a tool X need to developed which are written in C/C++ and having Tcl commanline interface, what will the steps or way?
I know about Tcl C API which can be used to extend Tcl by writing C extension for it.
What you're looking to do is embedding Tcl (totally a supported use case; Tcl remembers that it is a C library) but still making something tclsh-like. The simplest way of doing this is:
Grab a copy of tclAppInit.c (e.g., this is the current one in the Tcl 8.6 source tree as I write this) and adapt it, probably by putting the code to register your extra commands, linked variables, etc. in the Tcl_AppInit() function; you can probably trim a bunch of stuff out simply enough. Then build and link directly against the Tcl library (without stubs) to get effectively your own custom tclsh with your extra functionality.
You can use Tcl's API more extensively than that if you're not interested in interactive use. The core for non-interactive use is:
// IMPORTANT: Initialises the Tcl library internals!
Tcl_FindExecutable(argv[0]);
Tcl_Interp *interp = Tcl_CreateInterp();
// Register your custom stuff here
int code = Tcl_Eval(interp, "your script");
// Or Tcl_EvalFile(interp, "yourScriptFile.tcl");
const char *result = Tcl_GetStringResult(interp);
if (code == TCL_ERROR) {
// Really good idea to print out error messages
fprintf(stderr, "ERROR: %s\n", result);
// Probably a good idea to print error traces too; easier from in Tcl
Tcl_Eval(interp, "puts stderr $errorInfo");
exit(1);
}
// Print a non-empty result
if (result[0]) {
printf("%s\n", result);
}
That's about all you need unless you're doing interactive use, and that's when Tcl_Main() becomes really useful (it handles quite a few extra fiddly details), which the sample tclAppInit.c (mentioned above) shows how to use.
Usually, SWIG (Simplified Wrapper and Interface Generator) is the way to go.
SWIG HOMEPAGE
This way, you can write code in C/C++ and define which interface you want to expose.
suppose you have some C functions you want added to Tcl:
/* File : example.c */
#include <time.h>
double My_variable = 3.0;
int fact(int n) {
if (n <= 1) return 1;
else return n*fact(n-1);
}
int my_mod(int x, int y) {
return (x%y);
}
char *get_time()
{
time_t ltime;
time(<ime);
return ctime(<ime);
}
Now, in order to add these files to your favorite language, you need to write an "interface file" which is the input to SWIG. An interface file for these C functions might look like this :
/* example.i */
%module example
%{
/* Put header files here or function declarations like below */
extern double My_variable;
extern int fact(int n);
extern int my_mod(int x, int y);
extern char *get_time();
%}
extern double My_variable;
extern int fact(int n);
extern int my_mod(int x, int y);
extern char *get_time();
At the UNIX prompt, type the following:
unix % swig -tcl example.i
unix % gcc -fpic -c example.c example_wrap.c \
-I/usr/local/include
unix % gcc -shared example.o example_wrap.o -o example.so
unix % tclsh
% load ./example.so example
% puts $My_variable
3.0
% fact 5
120
% my_mod 7 3
1
% get_time
Sun Feb 11 23:01:07 2018
The swig command produces a file example_wrap.c that should be compiled and linked with the rest of the program. In this case, we have built a dynamically loadable extension that can be loaded into the Tcl interpreter using the 'load' command.
Taken from http://www.swig.org/tutorial.html
I need to get the address of a NULL void pointer. If I make a NULL c_void_p in Python I have no problem getting its address:
ptr = c_void_p(None)
print(ptr)
print(ptr.value)
print(addressof(ptr))
gives
c_void_p(None)
None
4676189120
But I have a
class Effect(structure):
_fields_ = [("ptr", c_void_p)]
where ptr gets initialized to NULL in C. When I access it in python
myclib.get_effect.restype = POINTER(Effect)
effect = myclib.get_effect().contents
print(effect.ptr)
gives None, so I can't take addressof(effect.ptr).
If I change my field type to a pointer to any ctype type
class Effect(structure):
_fields_ = [("ptr", POINTER(c_double)]
# get effect instance from C shared library
print(addressof(effect.ptr))
I have checked that I get the right address on the heap on the C side
140530973811664
Unfortunately, changing the field type from c_void_p is not an option. How can I do this?
Clarification
Here's C code following #CristiFati for my specific situation. struct is allocated in C, I get a ptr back to it in Python, and now I need to pass a reference to a ptr in the struct. First if I make the ptr a double, there's no problem!
#include <stdio.h>
#include <stdlib.h>
#define PRINT_MSG_2SX(ARG0, ARG1) printf("From C - [%s] (%d) - [%s]: ARG0: [%s], ARG1: 0x%016llX\n", __FILE__, __LINE__, __FUNCTION__, ARG0, (unsigned long long)ARG1)
typedef struct Effect {
double* ptr;
} Effect;
void print_ptraddress(double** ptraddress){
PRINT_MSG_2SX("Address of Pointer:", ptraddress);
}
Effect* get_effect(){
Effect* pEffect = malloc(sizeof(*pEffect));
pEffect->ptr = NULL;
print_ptraddress(&pEffect->ptr);
return pEffect;
}
And in Python
from ctypes import cdll, Structure, c_int, c_void_p, addressof, pointer, POINTER, c_double, byref
clibptr = cdll.LoadLibrary("libpointers.so")
class Effect(Structure):
_fields_ = [("ptr", POINTER(c_double))]
clibptr.get_effect.restype = POINTER(Effect)
pEffect = clibptr.get_effect()
effect = pEffect.contents
clibptr.print_ptraddress(byref(effect.ptr))
gives matching addresses:
From C - [pointers.c] (11) - [print_ptraddress]: ARG0: [Address of Pointer:], ARG1: 0x00007FC2E1AD3770
From C - [pointers.c] (11) - [print_ptraddress]: ARG0: [Address of Pointer:], ARG1: 0x00007FC2E1AD3770
But if I change the double* to void* and c_void_p, I get an error, because the c_void_p in python is set to None
ctypes ([Python 3]: ctypes - A foreign function library for Python) is meant to be able to "talk to" C from Python, which makes it Python friendly, and that means no pointers, memory addresses, ... whatsoever (well at least as possible, to be more precise).
So, under the hood, it does some "magic", which in this case stands between you and your goal.
#EDIT0: Updated the answer to better fit the (clarified) question.
Example:
>>> import ctypes
>>> s0 = ctypes.c_char_p(b"Some dummy text")
>>> s0, type(s0)
(c_char_p(2180506798080), <class 'ctypes.c_char_p'>)
>>> s0.value, "0x{:016X}".format(ctypes.addressof(s0))
(b'Some dummy text', '0x000001FBB021CF90')
>>>
>>> class Stru0(ctypes.Structure):
... _fields_ = [("s", ctypes.c_char_p)]
...
>>> stru0 = Stru0(s0)
>>> type(stru0)
<class '__main__.Stru0'>
>>> "0x{:016X}".format(ctypes.addressof(stru0))
'0x000001FBB050E310'
>>> stru0.s, type(stru0.s)
(b'Dummy text', <class 'bytes'>)
>>>
>>>
>>> b = b"Other dummy text"
>>> char_p = ctypes.POINTER(ctypes.c_char)
>>> s1 = ctypes.cast((ctypes.c_char * len(b))(*b), char_p)
>>> s1, type(s1)
(<ctypes.LP_c_char object at 0x000001FBB050E348>, <class 'ctypes.LP_c_char'>)
>>> s1.contents, "0x{:016X}".format(ctypes.addressof(s1))
(c_char(b'O'), '0x000001FBB050E390')
>>>
>>> class Stru1(ctypes.Structure):
... _fields_ = [("s", ctypes.POINTER(ctypes.c_char))]
...
>>> stru1 = Stru1(s1)
>>> type(stru1)
<class '__main__.Stru1'>
>>> "0x{:016X}".format(ctypes.addressof(stru1))
'0x000001FBB050E810'
>>> stru1.s, type(stru1.s)
(<ctypes.LP_c_char object at 0x000001FBB050E6C8>, <class 'ctypes.LP_c_char'>)
>>> "0x{:016X}".format(ctypes.addressof(stru1.s))
'0x000001FBB050E810'
This is a parallel between 2 types which in theory are the same thing:
ctypes.c_char_p: as you can see, s0 was automatically converted to bytes. This makes sense, since it's Python, and there's no need to work with pointers here; also it would be very annoying to have to convert each member from ctypes to plain Python (and viceversa), every time when working with it.
Current scenario is not part of the "happy flow", it's rather a corner case and there's no functionality for it (or at least I'm not aware of any)
ctypes.POINTER(ctypes.c_char) (named it char_p): This is closer to C, and offers the functionality you needed, but as seen it's also much harder (from Python perspective) to work with it
The problem is that ctypes.c_void_p is similar to #1., so there's no OOTB functionality for what you want, and also there's no ctypes.c_void to go with #2.. However, it is possible to do it, but additional work is required.
The well known (C) rule is: AddressOf(Structure.Member) = AddressOf(Structure) + OffsetOf(Structure, Member) (beware of memory alignment who can "play dirty tricks on your mind").
For this particular case, things couldn't be simpler. Here's an example:
dll.c:
#include <stdio.h>
#include <stdlib.h>
#if defined(_WIN32)
# define DLL_EXPORT __declspec(dllexport)
#else
# define DLL_EXPORT
#endif
#define PRINT_MSG_2SX(ARG0, ARG1) printf("From C - [%s] (%d) - [%s]: ARG0: [%s], ARG1: 0x%016llX\n", __FILE__, __LINE__, __FUNCTION__, ARG0, (unsigned long long)ARG1)
static float f = 1.618033;
typedef struct Effect {
void *ptr;
} Effect;
DLL_EXPORT void test(Effect *pEffect, int null) {
PRINT_MSG_2SX("pEffect", pEffect);
PRINT_MSG_2SX("pEffect->ptr", pEffect->ptr);
PRINT_MSG_2SX("&pEffect->ptr", &pEffect->ptr);
pEffect->ptr = !null ? NULL : &f;
PRINT_MSG_2SX("new pEffect->ptr", pEffect->ptr);
}
code.py:
#!/usr/bin/env python3
import sys
from ctypes import CDLL, POINTER, \
Structure, \
c_int, c_void_p, \
addressof, pointer
DLL = "./dll.dll"
class Effect(Structure):
_fields_ = [("ptr", c_void_p)]
def hex64_str(item):
return "0x{:016X}".format(item)
def print_addr(ctypes_inst, inst_name, heading=""):
print("{:s}{:s} addr: {:s} (type: {:})".format(heading, "{:s}".format(inst_name) if inst_name else "", hex64_str(addressof(ctypes_inst)), type(ctypes_inst)))
def main():
dll_dll = CDLL(DLL)
test_func = dll_dll.test
test_func.argtypes = [POINTER(Effect), c_int]
effect = Effect()
print_addr(effect, "effect")
test_func(pointer(effect), 1)
print(effect.ptr, type(effect.ptr)) # Not helping, it's Python int for c_void_p
try:
print_addr(effect.ptr, "effect.ptr")
except:
print("effect.ptr: - wrong type")
print_addr(effect, "effect", "\nSecond time...\n ")
print("Python addrs (irrelevant): effect: {:s}, effect.ptr: {:s}".format(hex64_str(id(effect)), hex64_str(id(effect.ptr))))
if __name__ == "__main__":
print("Python {:s} on {:s}\n".format(sys.version, sys.platform))
main()
Output:
(py35x64_test) e:\Work\Dev\StackOverflow\q053531795>call "c:\Install\x86\Microsoft\Visual Studio Community\2015\vc\vcvarsall.bat" x64
(py35x64_test) e:\Work\Dev\StackOverflow\q053531795>dir /b
code.py
dll.c
(py35x64_test) e:\Work\Dev\StackOverflow\q053531795>cl /nologo /DDLL /MD dll.c /link /NOLOGO /DLL /OUT:dll.dll
dll.c
Creating library dll.lib and object dll.exp
(py35x64_test) e:\Work\Dev\StackOverflow\q053531795>dir /b
code.py
dll.c
dll.dll
dll.exp
dll.lib
dll.obj
(py35x64_test) e:\Work\Dev\StackOverflow\q053531795>"e:\Work\Dev\VEnvs\py35x64_test\Scripts\python.exe" code.py
Python 3.5.4 (v3.5.4:3f56838, Aug 8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)] on win32
effect addr: 0x000001FB25B8CB10 (type: <class '__main__.Effect'>)
From C - [dll.c] (21) - [test]: ARG0: [pEffect], ARG1: 0x000001FB25B8CB10
From C - [dll.c] (22) - [test]: ARG0: [pEffect->ptr], ARG1: 0x0000000000000000
From C - [dll.c] (23) - [test]: ARG0: [&pEffect->ptr], ARG1: 0x000001FB25B8CB10
From C - [dll.c] (25) - [test]: ARG0: [new pEffect->ptr], ARG1: 0x00007FFFAFB13000
140736141012992 <class 'int'>
effect.ptr: - wrong type
Second time...
effect addr: 0x000001FB25B8CB10 (type: <class '__main__.Effect'>)
Python addrs (irrelevant): effect: 0x000001FB25B8CAC8, effect.ptr: 0x000001FB25BCC9F0
As seen, the address of effect is the same as the address of effect's ptr. But again, this is the simplest possible scenario. But, as explained a general solution, is preferred. However that's not possible, but it can be worked around:
Use the above formula and get the field offset using [SO]: Getting elements from ctype structure with introspection? (it's long, I had a hard time coming to the current solution - especially because of the 2 container types (Structure and Array) nesting possibilities; hopefully, it's bug free (or as close as possible) :) )
Modify the C interface to something like: Effect *get_effect(void **ptr), and store the address in the parameter
Modify the (Python) Effect structure, and instead of ctypes.c_void_p field have something that involves POINTER (e.g.: ("ptr", POINTER(c_ubyte))). The definition will differ from C, and semantically things are not OK, but at the end they're both pointers
Note: don't forget to have a function that destroys a pointer returned by get_effect (to avoid memory leaks)
So after raising this in the python bug tracker, Martin Panter and Eryk Sun provided a better solution.
There is indeed an undocumented offset attribute, which allows us to access the right location in memory without having to do any introspection. We can get back our pointer using
offset = type(Effect).ptr.offset
ptr = (c_void_p).from_buffer(effect, offset)
We can more elegantly wrap this into our class by using a private field and adding a property:
class Effect(Structure):
_fields_ = [("j", c_int),
("_ptr", c_void_p)]
#property
def ptr(self):
offset = type(self)._ptr.offset
return (c_void_p).from_buffer(self, offset)
I have added an integer field before our pointer so the offset isn't just zero. For completeness, here is the code above adapted with this solution showing that it works. In C:
#include <stdio.h>
#include <stdlib.h>
#define PRINT_MSG_2SX(ARG0, ARG1) printf("%s : 0x%016llX\n", ARG0, (unsigned long long)ARG1)
typedef struct Effect {
int j;
void* ptr;
} Effect;
void print_ptraddress(double** ptraddress){
PRINT_MSG_2SX("Address of Pointer:", ptraddress);
}
Effect* get_effect(){
Effect* pEffect = malloc(sizeof(*pEffect));
pEffect->ptr = NULL;
print_ptraddress(&pEffect->ptr);
return pEffect;
}
In Python (omitting the above Effect definition):
from ctypes import cdll, Structure, c_int, c_void_p, POINTER, byref
clibptr = cdll.LoadLibrary("libpointers.so")
clibptr.get_effect.restype = POINTER(Effect)
effect = clibptr.get_effect().contents
clibptr.print_ptraddress(byref(effect.ptr))
yields
Address of Pointer: : 0x00007F9EB248FB28
Address of Pointer: : 0x00007F9EB248FB28
Thanks again to everyone for quick suggestions. For more, see here:
I have several json files from a game I am modding that I want to be able to process using an external application.
as far as I can tell this code is within the specifications but I get a runtime error:
terminate called after throwing an instance of 'nlohmann::detail::type_error'
what(): [json.exception.type_error.304] cannot use at() with string
this is the stripped code to reproduce the error:
#include "json.hpp"
using json = nlohmann::json;
using namespace std;
namespace ns
{
class info
{
public:
std::string id;
};
void to_json(json& j, const info& mi);
void from_json(const json& j, info& mi);
}
int main()
{
json j ="[{\"id\": \"identifier\"}]";
ns::info info = j;
return 0;
}
void ns::to_json(json& j, const ns::info& mi)
{
j = json{
{"id",mi.id},
};
}
void ns::from_json(const json& j, ns::info& mi)
{
mi.id = j.at("id").get<std::string>();
}
and here is the compiler output:
-------------- Build: Debug in jsontest (compiler: GNU GCC Compiler)---------------
mingw32-g++.exe -Wall -std=c++11 -fexceptions -g -c C:\Users\UserOne\Documents\c++\jsontest\main.cpp -o obj\Debug\main.o
mingw32-g++.exe -o bin\Debug\jsontest.exe obj\Debug\main.o
Output file is bin\Debug\jsontest.exe with size 2.82 MB
Process terminated with status 0 (0 minute(s), 3 second(s))
0 error(s), 0 warning(s) (0 minute(s), 3 second(s))
There are two problems:
json j = "..."; initializes j with a JSON string value. It doesn't try to parse the contents. For that, you need to make it a json literal instead: json j = "..."_json;
After fixing that, you have a JSON array and but you're trying to access a field of a JSON object in ns::from_json().
So, fix both of those:
json j ="{\"id\": \"identifier\"}"_json;
and it'll work.
You might also consider using raw strings to avoid escaping all the quotes:
json j = R"({"id": "identifier"})"_json;
or just using an initializer list instead of parsing a string of json:
json j = { {"id", "identifier" } };
And if your source is a string being provided as a function argument or whatever, instead of a literal known at compile time:
std::string s = R"({"id": "identifier"})";
json j = json::parse(s);
I am using the thrust::copy_if function of the Thrust library coupled with counting iterators to get the indices of nonzero elements in an array. I also need to get the number of copied elements.
I am using the code from the 'counting_iterator.cu' example, except that in my application I need to reuse pre-allocated arrays, so I wrap them with thrust::device_ptr and then pass them to the thrust::copy_if function. This is the code:
using namespace thrust;
int output[5];
thrust::device_ptr<int> tp_output = device_pointer_cast(output);
float stencil[5];
stencil[0] = 0;
stencil[1] = 0;
stencil[2] = 1;
stencil[3] = 0;
stencil[4] = 1;
device_ptr<float> tp_stencil = device_pointer_cast(stencil);
device_vector<int>::iterator output_end = copy_if(make_counting_iterator<int>(0),
make_counting_iterator<int>(5),
tp_stencil,
tp_output,
_1 == 1);
int number_of_ones = output_end - tp_output;
If I comment the last line of code, the function fills correctly the output array. However, when I uncomment it, I get the following compilation error:
1>C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v5.5\include\thrust/iterator/iterator_adaptor.h(223): error : no operator "-" matches these operands
1> operand types are: int *const - const thrust::device_ptr
1> detected during:
1> instantiation of "thrust::iterator_adaptor::difference_type thrust::iterator_adaptor::distance_to(const thrust::iterator_adaptor &) const [with Derived=thrust::detail::normal_iterator>, Base=thrust::device_ptr, Value=thrust::use_default, System=thrust::use_default, Traversal=thrust::use_default, Reference=thrust::use_default, Difference=thrust::use_default, OtherDerived=thrust::device_ptr, OtherIterator=int *, V=signed int, S=thrust::device_system_tag, T=thrust::random_access_traversal_tag, R=thrust::device_reference, D=ptrdiff_t]"
1> C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v5.5\include\thrust/iterator/iterator_facade.h(181): here
1> instantiation of "Facade1::difference_type thrust::iterator_core_access::distance_from(const Facade1 &, const Facade2 &, thrust::detail::true_type) [with Facade1=thrust::detail::normal_iterator>, Facade2=thrust::device_ptr]"
1> C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v5.5\include\thrust/iterator/iterator_facade.h(202): here
1> instantiation of "thrust::detail::distance_from_result::type thrust::iterator_core_access::distance_from(const Facade1 &, const Facade2 &) [with Facade1=thrust::detail::normal_iterator>, Facade2=thrust::device_ptr]"
1> C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v5.5\include\thrust/iterator/iterator_facade.h(506): here
1> instantiation of "thrust::detail::distance_from_result, thrust::iterator_facade>::type thrust::operator-(const thrust::iterator_facade &, const thrust::iterator_facade &) [with Derived1=thrust::detail::normal_iterator>, Value1=signed int, System1=thrust::device_system_tag, Traversal1=thrust::random_access_traversal_tag, Reference1=thrust::device_reference, Difference1=signed int, Derived2=thrust::device_ptr, Value2=signed int, System2=thrust::device_system_tag, Traversal2=thrust::random_access_traversal_tag, Reference2=thrust::device_reference, Difference2=signed int]"
1> C:/ProgramData/NVIDIA Corporation/CUDA Samples/v5.5/7_CUDALibraries/nsgaIIparallelo_23ott/rank_cuda.cu(70): here
If I use thrust::device_vector for the output array instead, everything is okay:
using namespace thrust;
thrust::device_vector<int> output(5);
float stencil[5];
stencil[0] = 0;
stencil[1] = 0;
stencil[2] = 1;
stencil[3] = 0;
stencil[4] = 1;
device_ptr<float> tp_stencil = device_pointer_cast(stencil);
device_vector<int>::iterator output_end = copy_if(make_counting_iterator<int>(0),
make_counting_iterator<int>(5),
tp_stencil,
output.begin(),
_1 == 1);
int number_of_ones = output_end - output.begin();
Can you suggest any solution to this problem? Thank you.
Try using device_ptr instead of device_vector::iterator in your copy_if call:
thrust::device_ptr<int> output_end = copy_if(make_counting_iterator<int>(0),
make_counting_iterator<int>(5),
tp_stencil,
tp_output,
_1 == 1);