Convert semicolon-separated string into table structure? - csv

I need some help for converting a string into in ITAB.
LOOP AT LT_0111_FILE INTO LV_STRING.
SPLIT LV_STRING AT ';' INTO TABLE LT_0111.
DO GV_COMP TIMES.
READ TABLE LT_0111 ASSIGNING <LV> INDEX SY-INDEX.
IF <LV> IS NOT INITIAL.
ASSIGN COMPONENT SY-INDEX OF STRUCTURE <STRUCT> TO <LV_COMP>.
IF <LV_COMP> IS ASSIGNED.
<LV_COMP> = <LV>.
ENDIF.
ENDIF.
ENDDO.
INSERT <STRUCT> INTO TABLE <TABLE>.
ENDLOOP.
In LT_0111_FILE is the table PA0111 as a string with separator ";".
Now I need to assign every field of this stringtable into the fields of the structure from PA0111.
I donĀ“t want to do this for every field separatly because the fields will be created dynamically.
This code works for character fields but not for numbers. In the txt-file there will be numbers like 0,00 and to move them to the fields of the structure will give an error because the number have to be 0.00.
Thanks for the help

When you have an unknown structure and want to know what fields it has and what properties those fields have, then you can use the runtime type information classes.
First, get a type description of your target structure.
DATA lo_struct_description TYPE REF TO cl_abap_structdescr.
lo_struct_description ?= cl_abap_typedescr=>describe_by_data( <struct> ).
The casting-operator ?= is required here, because the return value of describe_by_data is a generic cl_abap_typedescr. You know that it got to be a structure, but the class does not know that. It could just as well be a table, simple type or a reference to an object. But if you can guarantee that it got to be a structure, you can up-cast it to a cl_abap_structdescr.
Now you can get a table describing all the fields of the structure with:
DATA lt_components TYPE abap_component_tab.
lt_components = lo_struct_description->get_components( ).
This table contains the names and the types of those components. So instead of using a DO-loop and using ASSIGN COMPONENT by index, you can LOOP AT the component table and use ASSIGN COMPONENT by name. And then you can handle each field according to its type:
LOOP AT lt_components INTO ls_component.
READ TABLE lt_0111 ASSIGNING <lv_in> INDEX sy-tabix.
IF sy-subrc = 0.
ASSIGN COMPONENT ls_component-name OF STRUCTURE <struct> TO <lv_out>.
IF sy-subrc = 0.
CASE ls_component-type->type_kind.
WHEN cl_abap_datadescr=>typekind_date.
" Special handling for dates
WHEN cl_abap_datadescr=>typekind_packed.
" Special handling for decimal numbers
" Check the other constants cl_abap_datadescr=>typekind_* for all the other types
" you might encounter in your use-case and which might require special treatment.
WHEN OTHERS.
" Probably just copyable. If not, you will get a runtime error here and need
" to implement special handling for this particular type_kind.
<lv_out> = <lv_in>.
ENDCASE.
ENDIF.
ENDIF.
ENDLOOP.

Actually I like Philips solution more, it is more error-proof and comprehensive in terms of type-handling, but just for the sake of diversity I will add this fast&dirty solution.
You can use methods methods of cl_rsda_csv_converter auxiliary class:
DATA: input TYPE TABLE OF string.
TYPES t_itab TYPE TABLE OF pa0009 WITH EMPTY KEY.
DATA(dref) = NEW t_itab( ).
APPEND ` 800;90051099;0;;;99991231;20080501;000;20100312;HORVATLU;;;;;;;;;;;;;;;46456.89;HUF;0.00;;0;;;;;HU;;;;;;;;;;;;;;;;00;;;;00000000;;;;; ` TO input.
APPEND ` 800;99000005;0;;;99991231;20170101;000;20170220;GUNASHMA;;;;;;;;;;;;;;;5564665.00;EUR;0.00;;0;;;;;DE;28511111;123;;;;;;;;;;;;;;00;;;;00000000;;;;; ` TO input.
DATA: ls_line TYPE LINE OF t_itab.
LOOP AT input ASSIGNING FIELD-SYMBOL(<fs_s>).
* creating converter
DATA(lo_csv) = cl_rsda_csv_converter=>create( i_separator = ';' ).
* Process records
lo_csv->csv_to_structure( EXPORTING i_data = <fs_s> IMPORTING e_s_data = ls_line ).
* inserting into itab
INSERT ls_line INTO TABLE dref->*.
ENDLOOP.

Related

convert json string to integer with pyspark

I want to convert a string object from json file into integer using pyspark.
df1.select(df1["`result.price`"]).dtypes
Out[15]: [('result.price', 'string')]
df1=df1.withColumn(df1.select(df1["`result.price`"]),F.col(df1.select(df1["`result.price`"])).cast(T.IntegerType()))
'DataFrame' object has no attribute '_get_object_id'
If you want to modify inline:
Since you are trying to modify the data type of nested struct field, I think you need to apply the new StructType.
Take a look at this https://stackoverflow.com/a/63270808/2956135
If you are okay with extracting to a different column:
df1 = df1.withColumn('price', F.col('result.price').cast(T.IntegerType()))
TL;DR
Why your line gives an error?
There is a few mistakes in this syntax.
df1 = df1.withColumn(df1.select(df1["`result.price`"]),F.col(df1.select(df1["`result.price`"])).cast(T.IntegerType()))
First, 1st argument of withColumn has to be string of a column name that you want to save as.
Second, F.col's argument has to be string of a column name or reference to the column.
So, this syntax should not throw an error, however, the casted value is saved to the new column.
df1 = df1.withColumn('result.price', F.col('result.price').cast(T.IntegerType()))

Unable to Extract simple Csv file using U-SQL

I have this csv file,
Almost all the records are getting processed fine, however there are two cases in which i am experiencing an issue.
Case 1:
A record containing quotes within quotes:
"some data "some data" some data"
Case 2:
A record containing comma within quotes:
"some data, some data some data"
i have looked into this issue, and got my way around looking into quoting parameter of the extractor, but i have observed that setting (quoting:false) solves case 1 and fails for case 2 and setting (quoting:true) solves case 2 but fails for case 1.
constraints: There is no room for changing the data file, the future data will be tailored accordingly but for this existing data i have to resolve this.
Try this, import records as one row and fix the row text using double quotes (do the same for the commas):
DECLARE #input string = #"/Samples/Data/Sample1.csv";
DECLARE #output string = #"/Output/Sample1.txt";
// Import records as one row
#data =
EXTRACT rowastext string
FROM #input
USING Extractors.Text('\n', quoting: false );
// Fix the row text using double quotes
#query =
SELECT Regex.Replace(rowastext, "([^,])\"([^,])", "$1\"\"$2") AS rowascsv
FROM #data;
OUTPUT #query
TO #output
USING Outputters.Csv(quoting : false);

Capture any standard report to JSON or XML?

I know that I can use LIST_TO_ASCI to convert a report to ASCII, but I would like to have a more high level data format like JSON, XML, CSV.
Is there a way to get something that is easier to handle then ASCII?
Here is the report I'd like to convert:
The conversion needs to be executed in ABAP on a result which was executed like this:
SUBMIT <REPORT_NAME> ... EXPORTING LIST TO MEMORY AND RETURN.
You can get access to SUBMIT list in memory like this:
call function 'LIST_FROM_MEMORY'
TABLES
listobject = t_list
EXCEPTIONS
not_found = 1
others = 2.
if sy-subrc <> 0.
message 'Unable to get list from memory' type 'E'.
endif.
call function 'WRITE_LIST'
TABLES
listobject = t_list
EXCEPTIONS
EMPTY_LIST = 1
OTHERS = 2
.
if sy-subrc <> 0.
message 'Unable to write list' type 'E'.
endif.
And the final step of the solution (conversion of result table to JSON) was already answered to you in your question.
I found a solution here: http://zevolving.com/2015/07/salv-table-22-get-data-directly-after-submit/
This is the code:
DATA: lt_outtab TYPE STANDARD TABLE OF alv_t_t2.
FIELD-SYMBOLS: <lt_outtab> like lt_outtab.
DATA lo_data TYPE REF TO data.
" Let know the model
cl_salv_bs_runtime_info=>set(
EXPORTING
display = abap_false
metadata = abap_false
data = abap_true
).
SUBMIT salv_demo_table_simple
AND RETURN.
TRY.
" get data from SALV model
cl_salv_bs_runtime_info=>get_data_ref(
IMPORTING
r_data = lo_data
).
ASSIGN lo_data->* to <lt_outtab>.
BREAK-POINT.
CATCH cx_salv_bs_sc_runtime_info.
ENDTRY.
Big thanks to Sandra Rossi, she gave me the hint to cx_salv_bs_sc_runtime_info.
Related answer: https://stackoverflow.com/a/52834118/633961

iterate through Poco::JSON::Object in insertion order

It is possible to preserve insertion order when parsing a JSON struct with a
Poco::JSON::Parser( new Poco::JSON::ParseHandler( true ) ): the non-default ParseHandler parameter preserveObjectOrder = true is handed over to the Poco::JSON::Objects so that they keep an private list of keys sorted in insertion order.
An object can then be serialized via Object::stringify() to look just like the source JSON string. Fine.
What, however, is the official way to step through a Poco::JSON::Object and access its internals in insertion order? Object::getNames() and begin()/end() use the alphabetical order of keys, not insertion order -- is there another way to access the values, or do I have to patch Poco?
As you already said:
Poco::JSON::ParseHandler goes into the Poco::JSON::Parser-constructor.
Poco::JSON::Parser::parse() creates a Poco::Dynamic::Var.
From that you'll extract a Poco::JSON::Object::Ptr.
The Poco::JSON:Object has the method "getNames". Beginning with this commit it seems to preserve the order, if it was requested via the ParseHandler. (Poco::JSON:Object::getNames 1.8.1, Poco::JSON:Object::getNames 1.9.0)
So now it should work as expected to use:
for(auto const & name : object->getNames()){
auto const & value = object->get(name); // or one of the other get-methods
// ... do things ...
}

Setting lua table in redis

I have a lua script, which simplified is like this:
local item = {};
local id = redis.call("INCR", "counter");
item["id"] = id;
item["data"] = KEYS[1]
redis.call("SET", "item:" .. id, cjson.encode(item));
return cjson.encode(item);
KEYS[1] is a stringified json object:
JSON.stringify({name : 'some name'});
What happens is that because I'm using cjson.encode to add the item to the set, it seems to be getting stringified twice, so the result is:
{"id":20,"data":"{\"name\":\"some name\"}"}
Is there a better way to be handling this?
First, regardless your question, you're using KEYS the wrong way and your script isn't written according to the guidelines. You should not generate key names in your script (i.e. call SET with "item:" .. id as a keyname) but rather use the KEYS input array to declare any keys involved a priori.
Secondly, instead of passing the stringified string with KEYS, use the ARGV input array.
Thirdly, you can do item["data"] = json.decode(ARGV[1]) to avoid the double encoding.
Lastly, perhaps you should learn about Redis' Hash data type - it may be more suitable to your needs.