Related
I'm importing webservice tests from Excel and serialising them as YAML.
But taking advantage of YAML being a superset of JSON I'd like the request part of the test to be valid JSON, i.e. to have delimeters, quotes and commas.
This will allow us to cut and paste requests between the automated test suite and manual test tools (e.g. Postman.)
So here's how I'd like a test to look (simplified):
- properties:
METHOD: GET
TYPE: ADDRESS
Request URL: /addresses
testCaseId: TC2
request:
{
"unitTypeCode": "",
"unitNumber": "15",
"levelTypeCode": "L",
"roadNumber1": "810",
"roadName": "HAY",
"roadTypeCode": "ST",
"localityName": "PERTH",
"postcode": "6000",
"stateTerritoryCode": "WA"
}
In Python, my request object has a dict attribute called fields which is the part of the object to be serialised as JSON. This is what I tried:
import yaml
def request_presenter(dumper, request):
json_string = json.dumps(request.fields, indent=8)
return dumper.represent_str(json_string)
yaml.add_representer(Request, request_presenter)
test = Test(...including embedded request object)
serialised_test = yaml.dump(test)
I'm getting:
- properties:
METHOD: GET
TYPE: ADDRESS
Request URL: /addresses
testCaseId: TC2
request: "{
\"unitTypeCode\": \"\",\n
\"unitNumber\": \"15\",\n
\"levelTypeCode": \"L\",\n
\"roadNumber1\": \"810\",\n
\"roadName\": \"HAY\",\n
\"roadTypeCode\": \"ST\",\n
\"localityName\": \"PERTH\",\n
\"postcode\": \"6000\",\n
\"stateTerritoryCode\": \"WA\"\n
}"
...only worse because it's all on one line and has white space all over the place.
I tried using the | style for literal multi-line strings which helps with the line breaks and escaped quotes (it's more involved but this answer was helpful.) However, escaped or multiline, the result is still a string that will need to be parsed separately.
How can I stop PyYaml analysing the JSON block as a string and make it just accept a block of text as part of the emitted YAML? I'm guessing it's something to do with overriding the emitter but I could use some help. If possible I'd like to avoid post-processing the serialised test to achieve this.
Ok, so this was the solution I came up with. Generate the YAML with a placemarker ahead of time. The placemarker marks the place where the JSON should be inserted, and also defines the root-level indentation of the JSON block.
import os
import itertools
import json
def insert_json_in_yaml(pre_insert_yaml, key, obj_to_serialise):
marker = '%s: null' % key
marker_line = line_of_first_occurrence(pre_insert_yaml, marker)
marker_indent = string_indent(marker_line)
serialised = json.dumps(obj_to_serialise, indent=marker_indent + 4)
key_with_json = '%s: %s' % (key, serialised)
serialised_with_json = pre_insert_yaml.replace(marker, key_with_json)
return serialised_with_json
def line_of_first_occurrence(basestring, substring):
"""
return line number of first occurrence of substring
"""
lineno = lineno_of_first_occurrence(basestring, substring)
return basestring.split(os.linesep)[lineno]
def string_indent(s):
"""
return indentation of a string (no of spaces before a nonspace)
"""
spaces = ''.join(itertools.takewhile(lambda c: c == ' ', s))
return len(spaces)
def lineno_of_first_occurrence(basestring, substring):
"""
return line number of first occurrence of substring
"""
return basestring[:basestring.index(substring)].count(os.linesep)
embedded_object = {
"unitTypeCode": "",
"unitNumber": "15",
"levelTypeCode": "L",
"roadNumber1": "810",
"roadName": "HAY",
"roadTypeCode": "ST",
"localityName": "PERTH",
"postcode": "6000",
"stateTerritoryCode": "WA"
}
yaml_string = """
---
- properties:
METHOD: GET
TYPE: ADDRESS
Request URL: /addresses
testCaseId: TC2
request: null
after_request: another value
"""
>>> print(insert_json_in_yaml(yaml_string, 'request', embedded_object))
- properties:
METHOD: GET
TYPE: ADDRESS
Request URL: /addresses
testCaseId: TC2
request: {
"unitTypeCode": "",
"unitNumber": "15",
"levelTypeCode": "L",
"roadNumber1": "810",
"roadName": "HAY",
"roadTypeCode": "ST",
"localityName": "PERTH",
"postcode": "6000",
"stateTerritoryCode": "WA"
}
after_request: another value
I'm getting this string as query result from my database:
"%Sample.Struct{list: [], total: \"0.00\", day: 6, id: \"8vfts6\"}"
Is there any way to convert this one back to map?
I'm getting this error decoding it with poison
** (Poison.SyntaxError) Unexpected token: %
(poison) lib/poison/parser.ex:56: Poison.Parser.parse!/2
(poison) lib/poison.ex:83: Poison.decode!/2
I can't fix the way data is being added to database, i must find a proper way for a key/value route to easily retrive data from that. (this is just a sample for a more complex result)
As it was mentioned in comments, you should not use Code.eval_string. But, there is a way to safely convert your code to Elixir struct, using Code module:
ex(1)> encoded = "%Sample.Struct{list: [], total: \"0.00\", day: 6, id: \"8vfts6\"}"
"%Sample.Struct{list: [], total: \"0.00\", day: 6, id: \"8vfts6\"}"
First, get the AST from the string, but use the pattern matching to ensure it is a struct you are looking for ({:__aliases__, _, [:Sample, :Struct]}). All other (potentially malicious) code will fail this match:
iex(2)> {:ok, {:%, _, [{:__aliases__, _, [:Sample, :Struct]}, {:%{}, _, keymap}]} = ast} = Code.string_to_quoted(encoded)
{:ok,
{:%, [line: 1],
[{:__aliases__, [line: 1], [:Sample, :Struct]},
{:%{}, [line: 1], [list: [], total: "0.00", day: 6, id: "8vfts6"]}]}}
Here you have the full ast for you struct, and the keymap. You may now be tempted to use eval_quoted with the AST, to get the struct you needed:
iex(3)> {struct, _} = Code.eval_quoted(ast)
{%Sample.Struct{day: 6, id: "8vfts6", list: [], total: "0.00"}, []}
iex(4)> struct
%Sample.Struct{day: 6, id: "8vfts6", list: [], total: "0.00"}
But it is still not safe! Someone may put a function causing side effect into the string, like "%Sample.Struct{list: IO.puts \"Something\"}", which will be executed during the evaluation. So you will need to check the keymap firsts, if it contain safe data.
Or you may just use keymap directly, without evaluating anyting:
iex(5)> struct(Sample.Struct, keymap)
%Sample.Struct{day: 6, id: "8vfts6", list: [], total: "0.00"}
I have seen something unusual with my SOAP UI script. I just want to perform an assertion that the data with is correct so I have written this code beLow:
import com.eviware.soapui.support.GroovyUtils
import groovy.json.JsonOutput
import groovy.json.JsonSlurper
def response = messageExchange.response.responseContent
def json = new JsonSlurper().parseText(response)
def jsonFormat = (response).toString()
def policies = [
[x: 28, xxx: 41, xxxxx: 1, name: 'Individual 18-50', aaa: true],
[x: 31, xxx: 41, xxxxx: 1, name: 'Individual 51-60', aaa: true],
[x: 34, xxx: 41, xxxxx: 1, name: 'Individual 61-75', aaa: true],
[x: 37, xxx: 41, xxxxx: 1, name: 'Individual 76-85', aaa: false]
]
log.warn json.policies
log.error policies
assert json.policies == policies
When I look at the log.warn and log.error info, it displays the json response in the incorrect order as it display 'isActive' field first.
log.warn json.policies displays this:
[{aaa=true, xx=28, xxxxx=1, name=Individual 18-50, xxxx=41}, {aaa=true, x=31, xxxxx=1, name=Individual 51-60, xxx=41}, {aaa=true, x=34, xxxxx=1, name=Individual 61-75, xxx=41}, {aaa=true, x=37, xxxxx=1, name=Individual 76-85, xxx=41}]
log.error policies displays this:
[{x=28, xxx=41, xxxxx=1, name=Individual 18-50, aaa=true}, {x=31, xxx=41, xxxxx=1, name=Individual 51-60, aaa=true}, {x=34, xxxx=41, xxxxxx=1, name=Individual 61-75, aaa=true}, {x=37, xxx=41, xxxxx=1, name=Individual 76-85, aaa=false}]
How can I have the DTOs to be displayed within the correct order with json.policies so that it is displayed in the correct order as policies?
One more unusual thing, I ran the test case 10 times and the test step which this assertion checks has passed 3 out of 10 times. It should never have passed as if you compare the last DTO as the end for policies, it displays isActive as false which the last isActive in json.policies is true.
You are almost close. But, couple of things to correct.
you do not have to convert json toString.
list should be sorted before comparison.
and it does not matter in which order each map entries are shown in the log, each map can be compared.
Here is the Script Assertion
//Check if the response is empty or not
assert context.response, 'Response is empty'
def json = new groovy.json.JsonSlurper().parseText(context.response)
//Assign the policies from current response; assuming above json contains that; change below statement otherwise.
def actualPolicies = json.policies
//Expected Polities
def expectedPolicies = [
[id: 28, providerId: 41, coverTypeId: 1, name: 'Individual 18-50', isActive: true],
[id: 31, providerId: 41, coverTypeId: 1, name: 'Individual 51-60', isActive: true],
[id: 34, providerId: 41, coverTypeId: 1, name: 'Individual 61-75', isActive: true],
[id: 37, providerId: 41, coverTypeId: 1, name: 'Individual 76-85', isActive: false]
]
log.info "List from response $actualPolicies"
log.info "List from policies $expectedPolicies"
//Sort the list and compare; each item is a map and using id to compare both
assert expectedPolicies.sort{it.id} == actualPolicies.sort{it.id}, 'Both policies are not matching'
You can quickly the online demo with fixed data based on your description to see it working the comparison.
I've looked at a few other related questions and I still can't seem to find what I'm looking for. This is an example JSON payload being sent to an API I'm writing:
{
"publishType": "Permitable",
"electricalPanelCapacity": 0.0,
"roofConstruction": "Standard/Pitched",
"roofType": "Composition Shingle",
"systemConstraint": "None",
"addedCapacity": 0.0,
"isElectricalUpgradeRequired": false,
"cadCompletedBy": "94039",
"cadCompletedDate": "2017-02-01T02:18:15Z",
"totalSunhourDeficit": 5.0,
"designedSavings": 5.0,
"isDesignedWithinTolerance": "N/A",
"energyProduction": {
"january": 322.40753170051255,
"february": 480.61501312589826,
"march": 695.35215022905118,
"april": 664.506907341219,
"may": 877.53769491124172,
"june": 785.56924358327,
"july": 782.64347308783363,
"august": 760.1123565793057,
"september": 574.67050827435878,
"october": 524.53797441350321,
"november": 324.31132291046379,
"december": 280.46921069200033
},
"roofSections": [{
"name": "North East Roof 4",
"roofType": "Composition Shingle",
"azimuth": 55.678664773137086,
"tilt": 15.0,
"solmetricEstimate": 510.42831656979456,
"shadingLoss": 14.0,
"systemRating": 580.0,
"sunHours": 0.88004882167205956,
"moduleCount": 2,
"modules": [{
"moduleRating": 290.0,
"isovaPartNumber": "CDS-MON-007070",
"partCount": 2
}]
}, {
"name": "South West Roof 3",
"roofType": "Composition Shingle",
"azimuth": 235.67866481720722,
"tilt": 38.0,
"solmetricEstimate": 3649.1643776261653,
"shadingLoss": 59.0,
"systemRating": 5220.0,
"sunHours": 0.69907363556056812,
"moduleCount": 18,
"modules": [{
"moduleRating": 290.0,
"isovaPartNumber": "CDS-MON-007070",
"partCount": 18
}]
}, {
"name": "South East Roof",
"roofType": "Composition Shingle",
"azimuth": 145.67866477313709,
"tilt": 19.0,
"solmetricEstimate": 2913.1406926526984,
"shadingLoss": 31.0,
"systemRating": 2900.0,
"sunHours": 1.0045312733285168,
"moduleCount": 10,
"modules": [{
"moduleRating": 290.0,
"isovaPartNumber": "CDS-MON-007070",
"partCount": 10
}]
}],
"SystemConfiguration": {
"inverters": [{
"isovaPartNumber": "ENP-INV-007182",
"partCount": 30
}]
}
}
Describing all the beginning parameters was easy.
/post/new-cad/{serviceNumber}:
post:
summary: Publish a new CAD record.
description: Creates a new CAD record under the provided service number and returns the name of the new CAD record, the unique SF ID, and the deep link URL for Salesforce.
parameters:
- name: serviceNumber
in: path
description: The service number for the solar project you're interested in publishing to.
required: true
type: string
- name: publishType
in: formData
description: The type of CAD record to publish (Proposal, Permitable, or AsBuilt).
required: true
type: string
- name: electricalPanelCapacity
in: formData
required: true
type: number
format: double
- name: roofConstruction
in: formData
description: New, Flat Roof, Open Beam, Standard/Pitched
required: true
type: string
- name: roofType
in: formData
description: Composition Shingle, Membrane (Rubber, TPO, PVC, EPDM), Metal - Corrugated (S-Curve), Metal - Standing Seam, Metal - Trapezoidal, Multi Roof Type, Rolled Comp, Silicone, Tar & Gravel, Tile - Flat, Tile - S-Curve, or Tile - W-Curve
type: string
- name: systemConstraint
in: formData
description: Usage, None, Roof, Electrical, Shading, or 10kW Max
required: true
type: string
- name: addedCapacity
in: formData
required: true
type: number
format: double
- name: isElectricalUpgradeRequired
in: formData
type: boolean
- name: cadCompletedBy
in: formData
description: Employee ID of record author.
type: number
required: true
- name: cadCompletedDate
in: formData
description: The date the CAD record was completed.
type: string
format: date
required: true
- name: totalSunhourDeficit
in: formData
type: number
format: double
- name: designedSavings
in: formData
type: number
format: double
- name: isDesignedWithinTolerance
in: formData
type: string
description: Yes, No, or N/A
And yields the expected result in Swagger-UI:
But now I'm struggling with the last parts of the example JSON payload above. I'm unsure how to express the energyProduction key which is an object with a key for each month of the year. I'm also unsure how to describe roofSections which is an array of objects and systemConfiguration which is an object with a property inverters whose value is an array of objects.
I'm going over the swagger documentation quite a bit but I'm still pretty confused and hoping maybe someone here can explain things a little better to me.
I figured it out. Turns out formData is not what I should have been using for my parameters. Instead I needed to use body and define the structure of the JSON that would populate the body. Here is the completed design file using a body parameter with an object schema and describes all the nested objects and arrays as well.
/new-cad/{serviceNumber}:
post:
summary: Publish a new CAD record.
description: Creates a new CAD record under the provided service number and returns the name of the new CAD record, the unique SF ID, and the deep link URL for Salesforce.
parameters:
- name: serviceNumber
in: path
description: The service number for the solar project you're interested in publishing to.
required: true
type: string
- name: cadData
in: body
description: A JSON payload containing the data required to publish a new CAD record.
required: true
schema:
type: object
properties:
publishType:
type: string
default: "Proposal"
enum: ["Proposal","Permitable","AsBuilt"]
electricalPanelCapacity:
type: number
roofConstruction:
type: string
default: "New"
enum: ["New","Flat Roof","Open Beam","Standard/Pitched"]
roofType:
type: string
enum: ["Composition Shingle","Membrane (Rubber, TPO, PVC, EPDM)","Metal - Corrugated (S-Curve)","Metal - Standing Seam","Metal - Trapezoidal","Multi Roof Type","Rolled Comp","Silicone","Tar & Gravel","Tile - Flat","Tile - S-Curve","Tile - W-Curve"]
systemConstraint:
type: string
default: "None"
enum: ["None","Usage","Roof","Electrical","Shading","10kW Max"]
addedCapacity:
type: number
default: 0
isElectricalUpgradeRequired:
type: boolean
cadCompletedBy:
type: string
cadCompletedDate:
type: string
totalSunhourDeficit:
type: number
designedSavings:
type: number
isDesignedWithinTolerance:
type: string
default: "N/A"
enum: ["N/A","Yes","No"]
energyProduction:
type: object
properties:
january:
type: number
february:
type: number
march:
type: number
april:
type: number
may:
type: number
june:
type: number
july:
type: number
august:
type: number
september:
type: number
october:
type: number
november:
type: number
december:
type: number
roofSections:
type: array
items:
type: object
properties:
name:
type: string
roofType:
type: string
enum: ["Composition Shingle","Membrane (Rubber, TPO, PVC, EPDM)","Metal - Corrugated (S-Curve)","Metal - Standing Seam","Metal - Trapezoidal","Multi Roof Type","Rolled Comp","Silicone","Tar & Gravel","Tile - Flat","Tile - S-Curve","Tile - W-Curve"]
azimuth:
type: number
tilt:
type: number
solmetricEstimate:
type: number
shadingLoss:
type: number
systemRating:
type: number
sunHours:
type: number
moduleCount:
type: integer
modules:
type: array
items:
type: object
properties:
moduleRating:
type: number
isovaPartNumber:
type: string
partCount:
type: integer
systemConfiguration:
type: object
properties:
inverters:
type: array
items:
type: object
properties:
isovaPartNumber:
type: string
partCount:
type: integer
tags:
- NEW-CAD
responses:
200:
description: CAD record created successfully.
schema:
type: object
properties:
cadName:
type: string
sfId:
type: string
sfUrl:
type: string
examples:
cadName: some name
sfId: a1o4c0000000GGAQA2
sfUrl: http://some-url-to-nowhere.com
204:
description: No project could be found for the given service number.
500:
description: Unexpected error. Most likely while communicating with Salesforce.
schema:
type: string
So now I can still get the serviceNumber from the path while everything else comes in the request body. One thing to keep in mind here is that you cannot use all the same Swagger Data Types. For example I tried to use double for one of the properties and Swagger complained that it couldn't parse type double. I was very confused until I finally found the section of the docs describing the difference between formData parameters and a body parameter (of which you can only have one, because it describes the entire request body). Basically you can only use data types that are supported by the JSON schema.
Swagger-UI now shows a single textarea instead of multiple input fields for each parameter. Not as pretty but it works great. You can click the "Example Value" box on the right and it places a predefined JSON template in the textarea for you so you can just fill in the values.
If you are just learning Swagger like I am I hope this helps!
I need to convert a JSON response into a well-formatted string, more like a table.
Example,
[{id: 1, name: "Panda", description: ""}, {id: 2, name: "koala", description: ""},...]
To be converted as,
Id | Name |Description
1 | Panda |
2 | Koala test |
and for the text to be automatically wrapped inside the cell width.
Any help would be appreciated.
Thank you.
First you need to parse JSON result.
data = JSON.parse(result)
Then use that parsed result to generate the pattern.
data = [
{ id: 1, name: 'Panda', description: '' },
{ id: 2, name: 'koala', description: '' }
]
puts 'Id | Name |Description'
data.each do |entry|
puts "#{entry[:id]} | #{entry[:name]} |#{entry[:description]}"
end
Hope this helps!