There is example piece of code from official website of skrape{it}. It is about extracting data from website. I'm using library version 1.0.0-alpha6
import it.skrape.core.htmlDocument
import it.skrape.selects.and
import it.skrape.selects.eachImage
import it.skrape.selects.eachText
import it.skrape.selects.html5.a
import it.skrape.selects.html5.div
import it.skrape.selects.html5.p
import it.skrape.selects.html5.span
import org.junit.jupiter.api.Test
// just some object where we will store our scraped data
data class MyExtractedData(
var httpMessage: String = "",
var userName: String = "",
var repositoryNames: List<String> = emptyList(),
var theThirdRepositoriesName: String = "",
var firstThreeHrefs: List<String> = emptyList(),
var overviewLink: String = "",
var firstThreeImageSources: List<String> = emptyList(),
var title: String = "",
var starsCount: String = ""
)
fun main() {
val extracted = skrape { // 1️⃣
url = "https://github.com/skrapeit"
extractIt<MyExtractedData> { it ->
it.httpMessage = status { message } // 2️⃣
htmlDocument { // 3️⃣
relaxed = true // 4️⃣
it.userName = ".h-card .p-nickname" { findFirst { text } } // 5️⃣
val repositories = span(".repo") { findAll { this }} // 6️⃣
println("hello world") // 7️⃣
it.repositoryNames = repositories.filter { it.text.contains("skrape") }.eachText // 8️⃣
it.theThirdRepositoriesName = span(".repo") {
2 { text } // 9️⃣
}
it.firstThreeImageSources = findAll { eachImage.map { image -> image.value } }.take(3) // 1️⃣0️⃣
it.firstThreeHrefs = findAll { eachHref }.take(3) // 1️⃣1️⃣
it.overviewLink = findAll { eachLink["Overview"] ?: "not found" } // 1️⃣2️⃣
it.title = titleText // 1️⃣3️⃣
// *️⃣
it.starsCount = div { // 1️⃣5️⃣
withClass = "pinned-item-list-item"
findFirst {
p { // 1️⃣6️⃣
findSecond {
a {
// 1️⃣7️⃣
withClass = "pinned-item-meta" and "muted-link" // 1️⃣8️⃣
withAttribute = "href" to "/skrapeit/skrape.it/stargazers" // 1️⃣9️⃣
findFirst {
ownText
}
}
}
}
}
}
}
}
}
println(extracted)
}
This piece of code simply does not compile. All possible imports had been done.
Errors:
Unresolved reference: status
Unresolved reference: message
Unresolved reference: relaxed
Function invocation 'eachText()' expected (line with 8️⃣ comment)
Expression '2' of type 'Int' cannot be invoked as a function. The function 'invoke()' is not found (line with 9️⃣ comment)
Type mismatch. Required: List. Found: List (line with 1️⃣0️⃣ comment)
And that's not all. There are lots of errors almost in every line. Where is the mistake?
Thank you!
I know I has been a while, but I was looking at the scrape{it} yesterday for the first time and face the same issue as you did.
The guys have an error on the official docs (probably it is just a way it used to work before).
To make it work, you need to change the top of the function slightly:
val extracted = skrape(HttpFetcher) { // 1️⃣
request {
url = "https://github.com/skrapeit"
}
It did a trick for me, hope will help someone as well.
I passed the following object:
var myVar = { typeA: { option1: "one", option2: "two" } }
I want to be able to pull out the key typeA from the above structure.
This value can change each time so next time it could be typeB.
So I would like to know if there is any way to do that
I was able to solve using 'keys'
for a json example like this:
{
"1-0001": {
"name": "red",
"hex": "FF0000"
},
"1-0002": {
"name": "blue",
"hex": "0000FF"
},
"1-0003": {
"name": "green",
"hex": "008000"
}
}
I was able to use
Map<String, dynamic> decoded = json.decode(jsonString);
for (var colour in decoded.keys) {
print(colour); // prints 1-0001
print(decoded[colour]['name']); // prints red
print(decoded[colour]['hex']); // prints FF0000
}
To get all filenames you can use:
var data = ...
var filenames = [];
for(var i = 0; i < data.length; i++) {
var item = data[0]['files'];
var key = item.keys.first;
var filename = item[key]['filename'];
filenames.add(filename);
}
print(filenames);
You need to define a data type.
It is basically a map of (key value-pair) where key is changed as stated in question typeA or typeB
This Object has 2 properties option1 and option2 which is also strings.
Here is the sample code to construct model and how to use it
import 'package:TestDart/TestDart.dart' as TestDart;
main(List<String> arguments) {
var map = new Map<String, MyObject>();
map['typeA'] = new MyObject("one", "two");
map['typeB'] = new MyObject("one", "two");
print(map['typeA'].toString());
print(map['typeA'].toString());
}
class MyObject {
String _option1;
String _option2;
MyObject(this._option1, this._option2);
String get option2 => _option2;
String get option1 => _option1;
#override
String toString() {
return 'MyObject{option1: $_option1, option2: $_option2}';
}
}
Relevant answer
map.forEach((key, value) {
print("Key : ${key} value ${value}");
});
I have one dropdown in the table.Once I make Onchange event I got all the table value and I want to convert it into json to send it to the servlet.
var item = [];
function dropDownOnChange(e) {
var selectedValue = e.options[e.selectedIndex].value;
alert("selectedValue:" + selectedValue);
var currentRow= $(e).closest("tr");
var AccountNo = $("td:eq(0)",$(currentRow)).text();
alert("accountno"+AccountNo);
var AccountType =$("td:eq(1)",$(currentRow)).text();
alert("acctyp"+AccountType);
var AcctypID = $("td:eq(3)",$(currentRow)).text();
alert("accID"+AcctypID);
Here I tried to convert it to JSON. I want to send this JSON value on my final save.
var objddlvalue = {};
objddlvalue["AccountNo"] = AccountNo;
objddlvalue["AccountType"] = AccountType;
objddlvalue["Account Type_Val"] = AcctypID;
objddlvalue["AccountStatus"] = selectedValue;
item.push(objddlvalue);
console.log(item);
jsonObj1 = JSON.stringify(item);
console.log(jsonObj1);
I am getting my JSON value like:
[{
"AccountNo": "89348734",
"AccountType": "Credit",
"Account Type_Val": "21",
"AccountStatus": "Invalid"
}]
When I check on online JSON checker it says the format is correct. But when I access it form servlet I can not parse it to jarray.
JSONObject jsonObj1 = (JSONObject)JSONValue.parse(request.getParameter("jsondata1"));
System.out.println("Json Object........"+jsonObj1.toJSONString());
JSONArray arr = (JSONArray) jsonObj.get(jsonObj1);
How to loop through my JSON object?
public static void main(String[] args) {
String jsonExternal = "[" +
"{"+
"\"AccountNo\": \"89348734\","+
"\"AccountType\": \"Credit\","+
"\"Account Type_Val\": \"21\","+
"\"AccountStatus\": \"Invalid\""+
"},"+
"{"+
"\"AccountNo\": \"89348734_test\","+
"\"AccountType\": \"Credit_test\","+
"\"Account Type_Val\": \"21_test\","+
"\"AccountStatus\": \"Invalid_test\""+
"}]";
JSONArray arr = (JSONArray)JSONValue.parse(jsonExternal);
for(Object obj : arr) {
JSONObject jsonObj = (JSONObject)obj;
Collection keySet = jsonObj.keySet();
Collection entrySet = jsonObj.entrySet();
Collection values = jsonObj.values();
for(Object o : keySet) {
System.out.println(o.toString());
}
for(Object o : entrySet) {
System.out.println(o.toString());
}
for(Object o : values) {
System.out.println(o.toString());
}
System.out.println(jsonObj.get("AccountNo"));
System.out.println(jsonObj.get("AccountType"));
System.out.println(jsonObj.get("Account Type_Val"));
System.out.println(jsonObj.get("AccountStatus"));
}
}
I haven't found a good library actionscript library for this yet.
I want to do things like:
Inflection.pluralize( "cat" ) == "cats"
Inflection.pluralize( "fish" ) == "fish"
I know ruby on rails has a library like this build in; and javascript has a really nice one as well: http://code.google.com/p/inflection-js/.
Any one know of a similar thing for actionscript?
Google says:
http://kuwamoto.org/2007/12/17/improved-pluralizing-in-php-actionscript-and-ror/
package
{
public class Inflect
{
private static var plural : Array = [
[/(quiz)$/i, "$1zes"],
[/^(ox)$/i, "$1en"],
[/([m|l])ouse$/i, "$1ice"],
[/(matr|vert|ind)ix|ex$/i, "$1ices"],
[/(x|ch|ss|sh)$/i, "$1es"],
[/([^aeiouy]|qu)y$/i, "$1ies"],
[/(hive)$/i, "$1s"],
[/(?:([^f])fe|([lr])f)$/i, "$1$2ves"],
[/(shea|lea|loa|thie)f$/i, "$1ves"],
[/sis$/i, "ses"],
[/([ti])um$/i, "$1a"],
[/(tomat|potat|ech|her|vet)o$/i, "$1oes"],
[/(bu)s$/i, "$1ses"],
[/(alias|status)$/i, "$1es"],
[/(octop)us$/i, "$1i"],
[/(ax|test)is$/i, "$1es"],
[/(us)$/i, "$1es"],
[/s$/i, "s"],
[/$/i, "s"]
];
private static var singular : Array = [
[/(quiz)zes$/i, "$1"],
[/(matr)ices$/i, "$1ix"],
[/(vert|ind)ices$/i, "$1ex"],
[/^(ox)en$/i, "$1"],
[/(alias|status)es$/i, "$1"],
[/(octop|vir)i$/i, "$1us"],
[/(cris|ax|test)es$/i, "$1is"],
[/(shoe)s$/i, "$1"],
[/(o)es$/i, "$1"],
[/(bus)es$/i, "$1"],
[/([m|l])ice$/i, "$1ouse"],
[/(x|ch|ss|sh)es$/i, "$1"],
[/(m)ovies$/i, "$1ovie"],
[/(s)eries$/i, "$1eries"],
[/([^aeiouy]|qu)ies$/i, "$1y"],
[/([lr])ves$/i, "$1f"],
[/(tive)s$/i, "$1"],
[/(hive)s$/i, "$1"],
[/(li|wi|kni)ves$/i, "$1fe"],
[/(shea|loa|lea|thie)ves$/i,"$1f"],
[/(^analy)ses$/i, "$1sis"],
[/((a)naly|(b)a|(d)iagno|(p)arenthe|(p)rogno|(s)ynop|(t)he)ses$/i, "$1$2sis"],
[/([ti])a$/i, "$1um"],
[/(n)ews$/i, "$1ews"],
[/(h|bl)ouses$/i, "$1ouse"],
[/(corpse)s$/i, "$1"],
[/(us)es$/i, "$1"],
[/s$/i, ""]
];
private static var irregular : Array = [
['move' , 'moves'],
['foot' , 'feet'],
['goose' , 'geese'],
['sex' , 'sexes'],
['child' , 'children'],
['man' , 'men'],
['tooth' , 'teeth'],
['person' , 'people']
];
private static var uncountable : Array = [
'sheep',
'fish',
'deer',
'series',
'species',
'money',
'rice',
'information',
'equipment'
];
public static function pluralize( string : String ) : String
{
var pattern : RegExp;
var result : String;
// save some time in the case that singular and plural are the same
if (uncountable.indexOf(string.toLowerCase()) != -1)
return string;
// check for irregular singular forms
var item : Array;
for each ( item in irregular )
{
pattern = new RegExp(item[0] + "$", "i");
result = item[1];
if (pattern.test(string))
{
return string.replace(pattern, result);
}
}
// check for matches using regular expressions
for each ( item in plural)
{
pattern = item[0];
result = item[1];
if (pattern.test(string))
{
return string.replace(pattern, result);
}
}
return string;
}
public static function singularize( string : String ) : String
{
var pattern : RegExp;
var result : String
// save some time in the case that singular and plural are the same
if (uncountable.indexOf(string.toLowerCase()) != -1)
return string;
// check for irregular singular forms
var item : Array;
for each ( item in irregular )
{
pattern = new RegExp(item[1] + "$", "i");
result = item[0];
if (pattern.test(string))
{
return string.replace(pattern, result);
}
}
// check for matches using regular expressions
for each ( item in singular)
{
pattern = item[0];
result = item[1];
if (pattern.test(string))
{
return string.replace(pattern, result);
}
}
return string;
}
public static function pluralizeIf(count : int, string : String) : String
{
if (count == 1)
return "1 " + string;
else
return count.toString() + " " + pluralize(string);
}
}
}
I have a complex JSON Object like this:
var requestData = { __batchRequests: [ { __changeRequests: [
{ requestUri: "Customers", method: "POST", headers: { "Content-ID": "1" }, data: {
CustomerID: 400, CustomerName: "John"
} }
] } ] };
I am trying to do two things:
Declare this object but with the variable data empty
With a loop, add items dynamically to the data object,
How can I do it?
This isn't too complex an object. And it isn't JSON until it's converted into a string.
Right now, it's just plain-ol' JS objects and arrays.
Breaking that down into its elements might look like this:
var requestData = {};
requestData.__batchRequests = [];
requestData.__batchRequests[0] = {};
requestData.__batchRequests[0].__changeRequests = [];
requestData.__batchRequests[0].__changeRequests[0] = {};
requestData.__batchRequests[0].__changeRequests[0].requestUri = "Customers";
requestData.__batchRequests[0].__changeRequests[0].method = "POST";
requestData.__batchRequests[0].__changeRequests[0].headers = { "Content-ID" : "1" };
requestData.__batchRequests[0].__changeRequests[0].data = {};
Aside from the repeats, what do you see?
Personally, I see that __changeRequests[0] is an object as simple as:
var changeRequest = {
requestUri : "Customers",
method : "POST",
headers : { "Content-ID" : "1" },
data : {}
};
I also see that I can just push that onto my array of change requests:
requestData.__batchRequests[0].__changeRequests.push(changeRequest);
Right?
I also know that my changeRequest variable still points to the one that I just added to the array, and whatever I change on the object will show up as changed in the array's reference to the object, too:
changeRequest.data.CustomerName = "Bob";
changeRequest.data.CustomerID = "204";
requestData.__/*...*/changeRequests[0].data.CustomerName; // Bob
So how about writing yourself some helper-functions?
function extend (obj, additions) {
var key;
for (key in obj) { if (additions.hasOwnProperty(key)) {
obj[key] = additions[key];
}
}
function makeChangeRequest (url, method, headers, data) {
var request = {
requestUri : url,
method : method,
headers : {},
data : {}
};
extend(request.headers, headers);
extend(request.data, data);
return request;
}
function getBatch (num) { return requestData.__batchRequests[num]; }
var changeReq = makeChangeRequest("Customers",
"POST",
{ "Content-ID" : "1" },
{ CustomerName : "Bob", CustomerID : "2012" });
var batch = getBatch(0);
batch.__changeRequests.push(changeReq);
If you want to add more data to changeReq.data later:
extend(changeReq.data, { Address : "33 Nowhere Rd.", City : "Splitsville" });
For the first part of your question, you can initialize data with an empty associative array:
var requestData = { __batchRequests: [ { __changeRequests: [
{ requestUri: "Customers", method: "POST", headers: { "Content-ID": "1" }, data: {} }
] } ] };
This next part assumes, perhaps incorrectly, that you can use jQuery. It also assumes that you have an array containing all of the relevant key value pairs.
var customerDeetsArray =[{CustomerID: 400}, {CustomerName: "John"}];
for (var i in customerDeetsArray) {
requestData.data = $.extend(requestData.data, customerDeetsArray[i]);
}
See working example which makes use of console.debug:
http://jsfiddle.net/4Rh72/6/