How to write table function with non fixed parameter list length?
Particular simplified example:
I want to write function trimupper(TableName,ColumnName1,ColumnName2,...) that combines just two steps for given set of columns:
TRIM whitespaces
UPPERCASE text
Example for two columns case:
(tbl as table, cn1 as text, cn2 as text) =>
let
#"Trimmed Text" = Table.TransformColumns(tbl,{{cn1, Text.Trim , type text}, {cn2, Text.Trim , type text}}),
#"Uppercased Text" = Table.TransformColumns(tbl,{{cn1, Text.Upper, type text}, {cn2, Text.Upper, type text}}),
trimupperResult = #"Uppercased Text"
in
trimupperResult
But how to do it for variable number of ColumnNames?
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
changethem = transform (Source,{"ColumnName1","ColumnName2"})
in changethem
function transform
(Table as table, columnnames as list) =>
let columnnames = if columnnames = null then Table.ColumnNames(Table) else columnnames,
change = Table.TransformColumns( Table, List.Transform(columnnames, each {_, Text.Trim, type text} ) ),
change1 = Table.TransformColumns( change, List.Transform(columnnames, each {_, Text.Upper, type text} ) )
in change1
Related
I need Some function in PowerQuery to get Additional Columns for duplicated data (not just keep/remove duplicates)
Example:
For the given table I want to get following info for duplicated columns set {"Date", "Product", "Color"}:
Minimal RowId - basicaly, Id of the 1st occurence of data
Nr. of Duplicate - duplicates counter within MinRowId group
NB! For non duplicates it should return null values
try grouping then expanding in powerquery
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Grouped Rows" = Table.Group(Source, {"Product", "Color"}, {
{"data", each Table.AddIndexColumn(_, "nDupl", 0, 1, Int64.Type), type table},
{"MinRowID", each List.Min(_[RowId]), type number}
}),
#"Expanded data" = Table.ExpandTableColumn(#"Grouped Rows", "data", {"RowId", "Date", "amount", "nDupl"}, {"RowId", "Date", "amount", "nDupl"})
in #"Expanded data"
Please try following function (download):
Function call Example:
tfnAddDuplicatesInfo2(Source,{"Product","Color","Date"},"DuplInfo" ,"RowId")
Function Arguments:
srcTable as table, // input Table
inGroupBy as list, // List of ColumnNames to search duplicates
outDuplInfo as text, // Output ColumnName for Information about Duplicates - Duplicate number and Minimal RowId (if inRowId provided) within a group
optional inRowId as nullable text // RowId ColumnName - required for outMinRowId calculation for inGroupBy columns
Function body:
let
func = (
srcTable as table, // input Table
inGroupBy as list, // List of ColumnNames to search duplicates
outDuplInfo as text, // Output ColumnName for Information about Duplicates - Duplicate number and Minimal RowId (if inRowId provided) within a group
optional inRowId as nullable text // RowId ColumnName - required for outMinRowId calculation for inGroupBy columns
) =>
let
Source = srcTable,
// // To test as script
// inGroupBy = {"Product", "Color","Date"},
// outDuplInfo = "DuplInfo",
// inRowId = "RowId", // null, "RowId",
//> == Variables ===================================================
Columns2Expand = List.Combine({List.Difference(Table.ColumnNames(Source),inGroupBy),{"__outDuplCounter__"}}),
srcType = Value.Type(Source),
srcTypeRow=
Type.ForRecord(
Record.Combine(
{
Type.RecordFields(Type.TableRow(srcType)),
Type.RecordFields(type [__outDuplCounter__= Int64.Type])
}
),
false
),
RowIdType = if inRowId<>null then Type.TableColumn(srcType,inRowId) else Any.Type, // Stores Column Typename
//< == Variables ===================================================
#"Grouped Rows" = Table.Group(
Source,
inGroupBy,
{
{"__tmpCount__" , each Table.RowCount(_), Int64.Type},
{"__MinGroupRowId__", each if inRowId<> null then List.Min( Record.Field(_,inRowId) ) else null, RowIdType},
{"__AllRows__" , each Table.AddIndexColumn(_, "__outDuplCounter__", 0, 1, Int64.Type), type table srcTypeRow}
}
),
#"Expanded __AllRows__" = Table.ExpandTableColumn(#"Grouped Rows", "__AllRows__", Columns2Expand),
nulls4MinRowId = Table.ReplaceValue(#"Expanded __AllRows__",each [__tmpCount__]<=1, null,
(currentValue, isConditionTrue, replacementValue) => if isConditionTrue then null else currentValue, // Replace.Value function
if inRowId<>null then {"__MinGroupRowId__","__outDuplCounter__"} else {"__outDuplCounter__"}
),
Add_outDuplInfo =
if inRowId<> null then
Table.AddColumn(nulls4MinRowId, outDuplInfo,
each
if [__outDuplCounter__]=null
then null
else [MinRowId=[__MinGroupRowId__], nDupl = [__outDuplCounter__]] ,
type nullable [MinRowId = RowIdType, nDupl = Int64.Type]
)
else
Table.AddColumn(nulls4MinRowId, outDuplInfo, each [__outDuplCounter__], Int64.Type),
Result_tfnAddDuplMinRowId = Table.SelectColumns(Add_outDuplInfo, List.Combine({Table.ColumnNames(Source),{outDuplInfo}}))
in
Result_tfnAddDuplMinRowId,
documentation = [
Documentation.Name = " tfnAddDuplicatesInfo2 ",
Documentation.Description = " Adds two info columns for Duplicates - 1st occurence RowId and given group Occurence Number",
Documentation.LongDescription = " Adds two info columns for Duplicates - 1st occurence RowId and given group Occurence Number",
Documentation.Category = " Running Total ",
Documentation.Source = " ",
Documentation.Version = " 1.0 ",
Documentation.Author = " Denis Sipchenko ",
Documentation.Examples = {
[
Description = "tfnAddDuplicatesInfo2 arguments: ",
Code = "
srcTable as table, // input Table
inGroupBy as list, // List of ColumnNames to search duplicates
outDuplInfo as text, // Output ColumnName for Information about Duplicates - Duplicate number and Minimal RowId (if inRowId provided) within a group
optional inRowId as nullable text // RowId ColumnName - required for outMinRowId calculation for inGroupBy columns",
Result =""
],
[
Description = "tfnAddDuplicatesInfo2 function call example ",
Code = "
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText(""hZTBasMwEET/xWdDdteSbP9CT4U2h2JyCK1oQ0xS3IT8frUpWsmSqpxs4ccw2pn1NDXYtA3CBsYNAZE7PNn96cc93+w8n2/uZWwBml07NfwVTIS+nN+PK1SDZzuW1RG7PX3Y5Wb3y4r3uHKHDgrSz9fle7buRQ2e1e5EpuA4sORZw+x/NgIvtnu2jbGP42G5rMS73sMDw0MdlhuODKua68Ai8KT7CH49fH5dVqOOaI6QoO5DCX1PkeraKDTnSKquLdNDjhGLvgMtsE6NZHUKrEnrVBPuU8/F0El6jRykox+UlSR45DCJamEGmODhhpERGNOa5BeNaErrna0NSU3ovpJjXVpqQip1LcGLbZSVJJ1OMLsjBtcm/Y8Ux43BCwcKxa0s0UPqPC84/hV89ws="", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [RowId = Int64.Type, Date = date, Product = _t, Color = _t, Amount = Currency.Type])
in
tfnAddDuplicatesInfo2(Source,{""Product"",""Color"",""Date""},""DuplInfo"" ,""RowId"")
",
Result = "Adds to Source table ""DuplInfo"" column with records:
""MinRowId"" - Minimal RowId within within given group,
""nDupl"" - given group Occurence Number
"
],
[
Description = "tfnAddDuplicatesInfo2 function short call example ",
Code = "
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText(""hZTBasMwEET/xWdDdteSbP9CT4U2h2JyCK1oQ0xS3IT8frUpWsmSqpxs4ccw2pn1NDXYtA3CBsYNAZE7PNn96cc93+w8n2/uZWwBml07NfwVTIS+nN+PK1SDZzuW1RG7PX3Y5Wb3y4r3uHKHDgrSz9fle7buRQ2e1e5EpuA4sORZw+x/NgIvtnu2jbGP42G5rMS73sMDw0MdlhuODKua68Ai8KT7CH49fH5dVqOOaI6QoO5DCX1PkeraKDTnSKquLdNDjhGLvgMtsE6NZHUKrEnrVBPuU8/F0El6jRykox+UlSR45DCJamEGmODhhpERGNOa5BeNaErrna0NSU3ovpJjXVpqQip1LcGLbZSVJJ1OMLsjBtcm/Y8Ux43BCwcKxa0s0UPqPC84/hV89ws="", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [RowId = Int64.Type, Date = date, Product = _t, Color = _t, Amount = Currency.Type])
in
tfnAddDuplicatesInfo2(Source,{""Product"",""Color"",""Date""},""nDupl"")
",
Result = "Adds to Source table one column:
""nDupl"" - given group Occurence Number
"
]
}
]
in
Value.ReplaceType(func, Value.ReplaceMetadata(Value.Type(func), documentation))
P.S. Idea about group & expand index column borrowed from horseyride post.
P.S.S. Initially, I took as a source Running Total by Category by Rick de Groot. And than reworked it.
I need to transform JSON.
I am getting output as "0000010006 GLAESS", but I need {"customer_id":"0000010006","customer":"GLAESS"}
TYPES: BEGIN OF ty_field,
customer_id TYPE string,
address TYPE string,
created_time TYPE string,
customer TYPE string,
date_created TYPE string,
END OF ty_field,
BEGIN OF ty_record,
id TYPE string,
createdtime TYPE string,
fields TYPE ty_field,
END OF ty_record,
BEGIN OF ty_response,
records TYPE STANDARD TABLE OF ty_record WITH EMPTY KEY,
END OF ty_response.
DATA:ls_response TYPE ty_response.
START-OF-SELECTION.
DATA(resp) = `{"records":[{"id":"rec5Qk24OQpKDyykq","createdTime":"2022-08-03T10:14:43.000Z","fields":{"customer_id":"0000010001","address":"Chennai","time_created":"06:00:14","customer":"IDADMIN","date_created":"16.04.2004"}},{"id":"rec7bSe8` &&
`Zb18z6b5a","createdTime":"2022-08-08T13:07:16.000Z","fields":{"customer_id":"0000010007","address":"Kakinada","time_created":"04:01:18","customer":"Ramya","date_created":"15.04.2000"}},{"id":"recD9Hh4YLgNXOhUE","createdTime":"2022-08-08T11:48:06.00` &&
`0Z","fields":{"customer_id":"0000010002","address":"Bangalore","time_created":"04:03:35","customer":"MAASSBERG","date_created":"20.04.2004"}},{"id":"recK7Tfw4PFAedDiB","createdTime":"2022-08-03T10:14:43.000Z","fields":{"customer_id":"0000010005","a` &&
`ddress":"Kakinada","time_created":"12:55","customer":"Lakshmi","date_created":"13-10-2022"}},{"id":"recKOq0DhEtAma7BV","createdTime":"2022-08-03T10:14:43.000Z","fields":{"customer_id":"0000010006","address":"Hyderabad","time_created":"18:42:28","cu` &&
`stomer":"GLAESS","date_created":"21.04.2004"}},{"id":"recS8pg10dFBGj8o7","createdTime":"2022-08-03T10:14:43.000Z","fields":{"customer_id":"0000010003","address":"Gurugram","time_created":"04:10:02","customer":"MAASSBERG","date_created":"20.04.2004"` &&
`}},{"id":"recf4QbOmKMrBeLQZ","createdTime":"2022-08-03T10:14:43.000Z","fields":{"customer_id":"0000010004","address":"Bangalore","time_created":"06:00:12","customer":"IDADMIN","date_created":"21.04.2004"}},{"id":"recs7oHEqfkN87tWm","createdTime":"2` &&
`022-08-03T10:14:43.000Z","fields":{"customer_id":"0000010000","address":"Hyderabad","time_created":"04:01:18","customer":"MAASSBERG","date_created":"15.04.2004"}}]}`.
/ui2/cl_json=>deserialize(
EXPORTING
json = resp
pretty_name = /ui2/cl_json=>pretty_mode-user
CHANGING
data = ls_response ).
DATA(ls_first_entry) = ls_response-records[ 5 ].
DATA(opt) = ls_first_entry-fields-customer_id && ` ` && ls_first_entry-fields-customer .
DATA(output) = /ui2/cl_json=>serialize(
data = opt
compress = abap_true
pretty_name = /ui2/cl_json=>pretty_mode-camel_case ).
If you want to serialize to a JSON object, you must use the corresponding ABAP structure. Currently you are serializing an abap string to json, that's why you get a simple json string.
TYPES: BEGIN OF ty_serialize,
customer_id TYPE string,
customer TYPE string,
END OF ty_serialize.
" Note: we build a structure, and not a string
DATA(ls_serialize) = VALUE ty_serialize( customer_id = ls_first_entry-fields-customer_id customer = ls_first_entry-fields-customer ).
DATA(lv_json1) = /ui2/cl_json=>serialize( data = ls_serialize
compress = abap_true
pretty_name = /ui2/cl_json=>pretty_mode-low_case ).
" Result: {"customer_id":"0000010006","customer":"GLAESS"}
I have a column called type that the value could be an object as a string, for example:
type = "{ 'a' : 'test1', 'b' : 'test2' }";
I want to get using Eloquent all rows where type.b = "test";
I can't access b in my where statement as it's not a column variable. I know I can use regex for it but I want to see if there is a better approach.
You should add "casts" in your Model like this
protected $casts = [
'type' => 'array'
];
And after it you can access b like this
$model->type['b'];
As a newbe, I have a question about Power Query (M)
I am looking for a way to extract samo info from an API result.
For starters I am doing this:
I have created a query to get the title from a task.
This works fine:
let
Source = Web.Contents(#fxGetSource() & "/tasks/IEABCDQ7KQPO5DQ4",
[Headers=[#"Authorization"=#fxGetHeader()]]),
convertToJson = Json.Document(Source),
data = convertToJson[data],
ConvertedToTable = Table.FromList(data, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
ExpandedColumn1 = Table.ExpandRecordColumn(ConvertedToTable, "Column1", {"title"}),
TheTitle = Table.TransformColumnTypes(ExpandedColumn1,{{"title", type text}})
in
TheTitle
I would like to have the taskid to sit in a variable, so I created a function:
(aTask as text) as text =>
let
Source = Web.Contents(#fxGetSource() & "/tasks/" & aTask,
[Headers=[#"Authorization"=#fxGetHeader()]]),
convertToJson = Json.Document(Source),
data = convertToJson[data],
ConvertedToTable = Table.FromList(data, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
ExpandedColumn1 = Table.ExpandRecordColumn(ConvertedToTable, "Column1", {"title"}),
TheTitle = Table.TransformColumnTypes(ExpandedColumn1,{{"title", type text}})
in
TheTitle
When I invoke this function ans use the taskid from above I get:
Expression Error: We cannot convert a value of type Table to type Text.
change
(aTask as text) as text =>
to
(aTask as text) as table =>
I found this answer that solves to one field -> Inserting multiple values into table with anorm
var fields: List[String] = Nil
var values: List[(String,ParameterValue[_])] = Nil
for ((username,i) <- usernames.zipWithIndex) {
fields ::= "({username%s})".format(i)
values ::= ("username" + i, username)
}
SQL("INSERT INTO users (username) VALUES %s".format(fields.mkString(",")))
.on(values: _*)
.executeUpdate()
How can I pass more fields, like username, address, phonenumber, etc?
I tried ...
def create(names: List[(String,ParameterValue[_])] ,addresses :List[(String,ParameterValue[_])]){
var fields: List[String] = Nil;
for((a,i) <- names.zipWithIndex){
fields ::= "({name%s},{address%s})".format(i)
}
DB.withConnection { implicit c =>
SQL("insert into table (name,address) values %s".format(fields.mkString(",")))
.on(names: _*, addresses: _*)
.executeUpdate()
}
}
I get the following error:
" no "_ *" annotation allowed here"
If I could use one single list to all parameters it'll even better.
You basically want to perform a batch insert. Here's an adaptation taken from the docs:
import anorm.BatchSql
val batch = BatchSql(
"INSERT INTO table (name, address) VALUES({username}, {address})",
Seq(names, addresses)
)
val res: Array[Int] = batch.execute() // array of update count