Unable to get json into an array - json

I have this data in a file named books.json that I am trying to get into an array.
{
"date_reviewed":"08/Jan/16",
"fname":"John",
"mname":"",
"lname":"Locke",
"email":"",
"title":"Boxed In!",
"imagelocation":"img\book_covers\BoxedIn.jpg",
"series_name":"Dr. Gideon Box",
"sellerssite":"https://www.amazon.com/Boxed-Gideon-Donovan-Creed-Novel-ebook/dp/B019HANKW8",
"genre":"Thriller,Suspence",
"authorsite":"https://johnlockeauthor.wordpress.com/",
"facebook":"https://www.facebook.com/john.locke.395669?fref=ts",
"fbid":"John Locke",
"twitter":"https://twitter.com/DonovanCreed",
"twitterid":"#DonovanCreed",
"review":"A series you have to be sick to love. I guess that's telling as I do love the series.
The latest in a series from an author that also has a number of other series that I have been devouring for the last couple of years. What makes these series unique is that they are all tied together by the characters that star in each series. So, if you're a fan of one and have not branched out into the others then it is time to go to his author page and start the reading adventure that will take you into the minds of a lethal assassin, a gunfighter and reluctant sheriff, and a private detective made famous by escaping from a deranged kidnapper.
Dr. Box may be brilliant when it comes to saving the life of children written off as lost causes but his activities outside the operating room have left any that know him with a strong desire to see him thrown into the bowels of hell as he surely is the very epitome of demon walking the earth. Good thing that he's the only one in the world that can save a life that others deem impossible. Just pray that if you've ever slighted him that you never show up in his hospital under his care. It will be an encounter you never live to regret.
In this tale Dr. Box is visited by the only man that causes true fear in a doctor who's moral compass was broken at birth. Donovan Creed. An assassin that you do not want to owe a favor to.
Donovan comes to collect. He has the plan to kill a terrorist and in the process save thousands of lives. The only problem, what he wants is the one thing Dr. Box cannot do.
Painted into a box (pun intended) he has no choice but to call on the one person that even Donovan is afraid of. Rose.
Together they come up with a full proof plan. Too bad that Donovan is already two steps ahead and the consequences for Dr. Box are steep and his life may never be the same.
As a fan, I say you have to add this to your library.",
"rating":"5"
}
I then retrieve then encode the data to escape the special characters in the data.
<?php
//Get data from existing json file
$jsondata = file_get_contents("books.json");;
$arr_data = json_encode($jsondata, true);
?>
I tried to then place that in an array but that failed so I thought maybe I had to decode it and place it into the array so I tried this code.
$Adata = json_decode($arr_data, true);
echo $Adata['fname'];
So putting that all together you have the following that produces and error "Warning: Illegal string offset 'fname'"
<?php
//Get data from existing json file
$jsondata = file_get_contents("books.json");;
$arr_data = json_encode($jsondata, true);
$Adata = json_decode($arr_data, true);
echo $Adata['fname'];
?>
I would appreciate any help. My ultimate goal is to get the data into an array so I can feed it to my code to instert it into my DB.

you code edit to this:
$jsondata = file_get_contents("books.json");
$arr_data = json_decode($jsondata, true);
echo $arr_data['fname']; //shoud print out: John

Related

How to convert string to json and vice versa in flutter?

Hi below is the json that I need to extract data from.
[
"http://activepeersai.computing.dcu.ie/feedback_participant/114",
"http://activepeersai.computing.dcu.ie/peerLearningPrompter/4",
"{\"0\":{\"feedback_id\":114,\"timer_used\":1,\"timer\":10.0,\"question1\":\"How to break ice with strangers (in social gathering & in formal events) ?\",\"question2\":\"How to build networking (with those from different age, culture, education background, language\\u2026)(in university & in workplace) ?\",\"question3\":\"\",\"question4\":\"\",\"question5\":\"\",\"question6\":\"\",\"question7\":\"\",\"question8\":\"\",\"question9\":\"\",\"question10\":\"\"},\"1\":{\"feedback_id\":115,\"timer_used\":1,\"timer\":10.0,\"question1\":\"How to deal with difficult teammates (dominating \\/ debater character \\/ negative \\/ lack of confidence \\/ free rider\\u2026) ?\",\"question2\":\"How to build mutual trust with teammates ?\",\"question3\":\"\",\"question4\":\"\",\"question5\":\"\",\"question6\":\"\",\"question7\":\"\",\"question8\":\"\",\"question9\":\"\",\"question10\":\"\"}}",
"{\"id\":{\"0\":1,\"1\":2,\"2\":3,\"3\":4,\"4\":5,\"5\":6,\"6\":7,\"7\":8,\"8\":9,\"9\":10,\"10\":11,\"11\":12,\"12\":13,\"13\":14,\"14\":15,\"15\":16,\"16\":17,\"17\":18,\"18\":19,\"19\":20,\"20\":21,\"21\":22,\"22\":23,\"23\":24,\"24\":25,\"25\":26,\"26\":27,\"27\":28,\"28\":29,\"29\":30,\"30\":31,\"31\":32,\"32\":33,\"33\":34,\"34\":35,\"35\":40,\"36\":41,\"37\":42,\"38\":43,\"39\":44,\"40\":45,\"41\":46,\"42\":47,\"43\":48,\"44\":49,\"45\":50,\"46\":51,\"47\":52,\"48\":53,\"49\":54,\"50\":55,\"51\":56,\"52\":57,\"53\":58,\"54\":59,\"55\":60,\"56\":61,\"57\":62,\"58\":63,\"59\":36,\"60\":37,\"61\":38,\"62\":39},\"name\":{\"0\":\"Why did the sharer choose this skill as the most confident skill and why did the learner wants to improve this skill?\",\"1\":\"Can you share another situation whereby using this skill made a very big difference to the outcome? What did you do? What was the result? What might have happened if this skill wasn't brought to the situation?\",\"2\":\"What advice do you have to give to others about becoming better at using this skill in their work and in their lives generally?\",\"3\":\"What can hold people back from being better at this skill? How can you encourage them to overcome these challenges?\",\"4\":\"What would be the outcome if this session\\/series was a huge success? How would we know if this happened?\",\"5\":\"What is working for you now? What is stopping you from moving forward?\",\"6\":\"What do your role models do that you would love to learn\\/incorporate as habits\\/adopt as a mindset?\",\"7\":\"\\\"What got you here won't get you there.\\\" What is your reaction to that statement? What actions have got you to this point that may not serve you if you move forward? What new behaviours do you need to adopt?\",\"8\":\"What would be the outcome if this session\\/series was a huge success? How would we know if this happened?\",\"9\":\"If you had all the time, people, money, resources that you might possibly need, what would you do differently? Does that energise you, frighten you or a bit of both?\",\"10\":\"What do you think you need to do get a better result (or closer to your goal)?\",\"11\":\"What is the worst that could happen and how could you handle it? What is the best that could happen and how could you handle it?\",\"12\":\"What have been the most impactful decisions that you've made in your career? What led to making those decisions? Were you aware of the impact those decisions would have? What would you say to others in a similar position?\",\"13\":\"What habits have stood you in good stead? Are there things that you do regularly, daily or very often that have made a big different over time?\",\"14\":\"Can you share an experience that was transformational for you, taught you a long-lasting lesson or was particularly memorable for its benefits and challenges?\",\"15\":\"How do you think experience affects somebody's perspective, way of making decisions or their feelings about taking risks?\",\"16\":\"Can you describe your target market for your business or who might your ideal employer be?\",\"17\":\"Can you share details of a project that you worked on recently? Why were you or your business chosen to work with this client\\/employer? What were the success and challenges along the way?\",\"18\":\"What makes you\\/your product\\/your service different to others? Why would a client or employer choose what you do over others?\",\"19\":\"What might I be able to do to help you?\",\"20\":\"What is the key problem that you are facing currently?\",\"21\":\"If that problem was solved, what impact would it have?\",\"22\":\"In order to get to that outcome, what do we need to do, what resources need to be invested and any other changes to be made?\",\"23\":\"Is there a willingness to take that action to get towards that outcome?\",\"24\":\"What assumptions are being made? How likely are each one to happen? What would be the impact of those assumptions happening or not happening?\",\"25\":\"What application of the SCAMPER technique could be useful to help get another perspective?\\n*SCAMPER is an acronym formed from the abbreviation of: Substitute, Combine, Adapt, Modify (Also magnify and minify), Put to another use, Eliminate and Reverse.\",\"26\":\"If we had double the budget, half the time and were living in another country trying to make this happen, how might we think differently?\",\"27\":\"If any idea generated already in this discussion was to become a reality, what would be the impact on e year later in terms of benefits, ongoing coasts, sustained behavioural change and a launchpad for further growth?\",\"28\":\"Leadership\",\"29\":\"Communication\",\"30\":\"Adaptability\",\"31\":\"Team Work\",\"32\":\"Problem Solving\",\"33\":\"Conflict Management\",\"34\":\"Productivity\",\"35\":\"How would you describe good\\/effective communication? Please share an example of a time that you've seen it in action and an example of when you saw that good communication skills were clearly lacking.\",\"36\":\"How have you handled working under someone you felt was not good at communicating?\",\"37\":\"If you're trying to get your point across or convince somebody that your idea is the right one, what do you do?\",\"38\":\"Who do you think is a good communicator and why? What can we learn from them?\",\"39\":\"Talk about a time that you needed to adapt to a new situation. What did you find difficult and how did you work through that? How can somebody prepare to be more adaptable in future situations?\",\"40\":\"When you're in a situation where it feels like you have no control over it (i.e. a new manager, starting in a new job, government-led changes etc), what do you do to focus on what you can do?\",\"41\":\"How do you handle having multiple priorities at the same time?\",\"42\":\"How do you adjust to different work settings? For example, working with different teams, switching between logic and creativity, learning new processes, tools or technologies?\",\"43\":\"How do you feel about working in a team? What do you think are the key things that need to happen to make good teamwork?\",\"44\":\"What has been your experience of working in teams where there were problems? Did these arise due to strong personalities, somebody not sharing the workload, miscommunication, the wrong support or technology systems etc?\",\"45\":\"How do you keep a team motivated? Share your story about a rewarding team experience.\",\"46\":\"When you're in a team situation, what role do you usually play?\",\"47\":\"Describe a situation where you had to solve a problem. What did you do? what was the result? What might you have done differently?\",\"48\":\"What steps do you take before making a decision on how to solve a problem, and why?\",\"49\":\"Give an example of a situation in which you saw an opportunity in a potential problem. What did you do? What was the outcome?\",\"50\":\"Can you tell me about a situation where you overcame a problem using a creative solution?\",\"51\":\"Have you ever had a team member who kept raising objections on projects? How did you (or would you) manage them?\",\"52\":\"You have noticed that a team member is aggressive or arrogant toward the rest of the team. How would you approach this person?\",\"53\":\"What would you do if your manager gave you negative feedback on the way you approached a problem? How do give negative\\/constructive feedback to others?\",\"54\":\"How could you use a situation with conflict to have a better relationship with all involved?\",\"55\":\"How would you describe a typical working day in your current role? How you manage importance versus urgency? How do you maintain a work-life balance also?\",\"56\":\"If you have your day planned out to achieve a goal, how do you manage distractions or other things that can happen along the way?\",\"57\":\"What do you think very productive people do differently than others?\",\"58\":\"What holds people back from being more productive? How can people turn those around so that by doing the opposite, they can become more productive?\",\"59\":\"Have you ever had a team member who kept raising objections on projects? How did you (or would you) manage them?\",\"60\":\"How do you describe your leadership style?\",\"61\":\"What was a difficult decision you had to make as a leader, and how did you come to that decision?\",\"62\":\"What are the most important attributes of successful leaders today? Who do you think is a good leader and why?\"},\"session_type_id\":{\"0\":1.0,\"1\":1.0,\"2\":1.0,\"3\":1.0,\"4\":2.0,\"5\":2.0,\"6\":2.0,\"7\":2.0,\"8\":3.0,\"9\":3.0,\"10\":3.0,\"11\":3.0,\"12\":4.0,\"13\":4.0,\"14\":4.0,\"15\":4.0,\"16\":5.0,\"17\":5.0,\"18\":5.0,\"19\":5.0,\"20\":6.0,\"21\":6.0,\"22\":6.0,\"23\":6.0,\"24\":7.0,\"25\":7.0,\"26\":7.0,\"27\":7.0,\"28\":8.0,\"29\":8.0,\"30\":8.0,\"31\":8.0,\"32\":8.0,\"33\":8.0,\"34\":8.0,\"35\":null,\"36\":null,\"37\":null,\"38\":null,\"39\":null,\"40\":null,\"41\":null,\"42\":null,\"43\":null,\"44\":null,\"45\":null,\"46\":null,\"47\":null,\"48\":null,\"49\":null,\"50\":null,\"51\":null,\"52\":null,\"53\":null,\"54\":null,\"55\":null,\"56\":null,\"57\":null,\"58\":null,\"59\":null,\"60\":null,\"61\":null,\"62\":null},\"session_type_name\":{\"0\":\"Learning and Development\",\"1\":\"Learning and Development\",\"2\":\"Learning and Development\",\"3\":\"Learning and Development\",\"4\":\"Mentoring\",\"5\":\"Mentoring\",\"6\":\"Mentoring\",\"7\":\"Mentoring\",\"8\":\"Coaching\",\"9\":\"Coaching\",\"10\":\"Coaching\",\"11\":\"Coaching\",\"12\":\"Experience Sharing\",\"13\":\"Experience Sharing\",\"14\":\"Experience Sharing\",\"15\":\"Experience Sharing\",\"16\":\"Networking\",\"17\":\"Networking\",\"18\":\"Networking\",\"19\":\"Networking\",\"20\":\"Establishing Buy-In\",\"21\":\"Establishing Buy-In\",\"22\":\"Establishing Buy-In\",\"23\":\"Establishing Buy-In\",\"24\":\"Idea Exploration\",\"25\":\"Idea Exploration\",\"26\":\"Idea Exploration\",\"27\":\"Idea Exploration\",\"28\":\"Seven Soft Skills\",\"29\":\"Seven Soft Skills\",\"30\":\"Seven Soft Skills\",\"31\":\"Seven Soft Skills\",\"32\":\"Seven Soft Skills\",\"33\":\"Seven Soft Skills\",\"34\":\"Seven Soft Skills\",\"35\":null,\"36\":null,\"37\":null,\"38\":null,\"39\":null,\"40\":null,\"41\":null,\"42\":null,\"43\":null,\"44\":null,\"45\":null,\"46\":null,\"47\":null,\"48\":null,\"49\":null,\"50\":null,\"51\":null,\"52\":null,\"53\":null,\"54\":null,\"55\":null,\"56\":null,\"57\":null,\"58\":null,\"59\":null,\"60\":null,\"61\":null,\"62\":null},\"parent_id\":{\"0\":null,\"1\":null,\"2\":null,\"3\":null,\"4\":null,\"5\":null,\"6\":null,\"7\":null,\"8\":null,\"9\":null,\"10\":null,\"11\":null,\"12\":null,\"13\":null,\"14\":null,\"15\":null,\"16\":null,\"17\":null,\"18\":null,\"19\":null,\"20\":null,\"21\":null,\"22\":null,\"23\":null,\"24\":null,\"25\":null,\"26\":null,\"27\":null,\"28\":null,\"29\":null,\"30\":null,\"31\":null,\"32\":null,\"33\":null,\"34\":null,\"35\":30.0,\"36\":30.0,\"37\":30.0,\"38\":30.0,\"39\":31.0,\"40\":31.0,\"41\":31.0,\"42\":31.0,\"43\":32.0,\"44\":32.0,\"45\":32.0,\"46\":32.0,\"47\":33.0,\"48\":33.0,\"49\":33.0,\"50\":33.0,\"51\":34.0,\"52\":34.0,\"53\":34.0,\"54\":34.0,\"55\":35.0,\"56\":35.0,\"57\":35.0,\"58\":35.0,\"59\":29.0,\"60\":29.0,\"61\":29.0,\"62\":29.0}}"
]
I need to extract data from the 3rd line.
The below code tries to extract the first set of questions from the 3rd line.
import 'dart:core';
import 'dart:convert';
import 'package:http/http.dart' as http;
class Question {
late final String question;
Question({ required this.question});
}
void getData() async{
List<Question> sample_data = [];
String link = 'https://activepeersai.computing.dcu.ie/API/631';
var url = Uri.parse(link);
var result = await http
.get(url,headers: {"Accept": "application/json","Access-Control_Allow_Origin": "*"});
if(result.statusCode==200){
var data = jsonDecode(result.body);
var res = jsonDecode(data[2]);
String round1 = json.encode(res);
var data1 = jsonDecode(round1);
var questions = jsonDecode(data1[0]);
for(var k in questions.keys){
if(((k=='question1')|| (k=='question2') || (k=='question3') || (k=='question4') || (k=='question5') || (k=='question6') || (k=='question7') || (k=='question8') || (k=='question9') || (k=='question10'))){
Question question = Question(question: res[k]);
sample_data.add(question);
}
}
print('$sample_data');
}
}
void main() {
getData();
}
Now when I print round1 I get the below output.
{"0":{"feedback_id":114,"timer_used":1,"timer":10,"question1":"How to break ice with strangers (in social gathering & in formal events) ?","question2":"How to build networking (with those from different age, culture, education background, language…)(in university & in workplace) ?","question3":"","question4":"","question5":"","question6":"","question7":"","question8":"","question9":"","question10":""},"1":{"feedback_id":115,"timer_used":1,"timer":10,"question1":"How to deal with difficult teammates (dominating / debater character / negative / lack of confidence / free rider…) ?","question2":"How to build mutual trust with teammates ?","question3":"","question4":"","question5":"","question6":"","question7":"","question8":"","question9":"","question10":""}}
Which when I checked on Json Checker came to be valid.
But after this I get an error.
Error: Expected a value of type 'String', but got one of type 'Null'
at Object.throw_ [as throw] (http://localhost:56003/dart_sdk.js:5080:11)
at Object.castError (http://localhost:56003/dart_sdk.js:5039:15)
at Object.cast [as as] (http://localhost:56003/dart_sdk.js:5356:17)
at String.as (http://localhost:56003/dart_sdk.js:46240:19)
at getData (http://localhost:56003/packages/testing_for_data/main.dart.lib.js:74:56)
at getData.next (<anonymous>)
at http://localhost:56003/dart_sdk.js:40641:33
at _RootZone.runUnary (http://localhost:56003/dart_sdk.js:40511:59)
at _FutureListener.thenAwait.handleValue (http://localhost:56003/dart_sdk.js:35438:29)
at handleValueCallback (http://localhost:56003/dart_sdk.js:35999:49)
at _Future._propagateToListeners (http://localhost:56003/dart_sdk.js:36037:17)
at [_completeWithValue] (http://localhost:56003/dart_sdk.js:35872:23)
at async._AsyncCallbackEntry.new.callback (http://localhost:56003/dart_sdk.js:35906:35)
at Object._microtaskLoop (http://localhost:56003/dart_sdk.js:40778:13)
at _startMicrotaskLoop (http://localhost:56003/dart_sdk.js:40784:13)
at http://localhost:56003/dart_sdk.js:36261:9
Can someone tell me what I am doing wrong?
Its coming due to the line
var data1 = jsonDecode(round1);
I believe this line is wrong
var questions = jsonDecode(data1[0]);
Firstly, because you probably need data1['0'] instead of data1[0]. They key is a string, not a number.
Secondly. You don't need to decode it again. It's already decoded. So try
var questions = data1['0'];
The next thing is that
Question question = Question(question: res[k]);
needs to be changed to
Question question = Question(question: questions[k]);
Because you actually want it from the questions.
If you want it to print the question instead of Instance of 'Question' you can add a toString method to it, for example:
class Question {
late final String question;
Question({ required this.question});
#override
String toString() {
return question;
}
}

Cannot retrieve field from a JSON object return by amazon-product-api

Hello every body I am new node.js I have this code `
client.itemSearch({
keywords:'borderlands 2' ,
searchIndex: 'VideoGames',
responseGroup: 'ItemAttributes,Images'
}).then(function(results){
res.end(JSON.stringify(results[0].ItemAttributes));
var kson = JSON.parse(JSON.stringify(results[0].ItemAttributes));
console.log(JSON.stringify(kson['Title']));
}).catch(function(err){
console.log(err);
});`
The json result of the amazon search is this [{"Binding":["Video Game"],"Brand":["2K"],"CatalogNumberList":[{"CatalogNumberListElement":["49101"]}],"Creator":[{"_":"Take 2","$":{"Role":"Creator"}}],"Department":["Video Games"],"EAN":["0163120331895"],"EANList":[{"EANListElement":["0163120331895","0151903126141","0031111574449","0088021472273","0021113169989","0100177413464","7104254910160","0710425491016","0745449851300"]}],"Edition":["Standard"],"ESRBAgeRating":["Mature"],"Feature":["Dynamic Co-op online, split-screen and LAN: Share your adventures with friends both online and via LAN. Borderlands 2 features a seamless system enabling you to drop in and drop out of a campaign without ever having to restart the game.","World Connected Story: You will find yourself left for dead in the frozen tundra of Pandora as you begin your quest of revenge and redemption.","New Badass Gun System: An entirely new take on the groundbreaking procedural system means millions upon millions of possible weapons. The variety and style of guns in Borderlands 2 is staggering.","87 Bazillion Everything: In addition to the new gun system, you will lust after procedurally generated shields, grenades, Alien Artifacts, class mods and much, much more. And you thought the original Borderlands had a ton of loot!","Brand new environments on Pandora: Hunt through entirely brand new areas of Pandora that are more alive than ever!","Choose from all new characters and classes including the Gunzerker, who can dual-wield any two weapons in the game","Share your adventures with friends both online and via LAN with a seamless co-op system enabling you to drop in and drop out of a campaign without ever having to restart the game","Improved story integration which finds you left for dead in the frozen tundra of Pandora as you begin your quest of revenge and redemption","An entirely new take on the groundbreaking procedural system means millions upon millions of possible weapons","In addition to the new gun system, players lust after procedurally generated shields, grenades, alien artifacts, class mods and much, much more"],"Genre":["shooter-action-game-genre"],"HardwarePlatform":["Xbox 360"],"IsAdultProduct":["0"],"IsAutographed":["0"],"IsEligibleForTradeIn":["1"],"IsMemorabilia":["0"],"ItemDimensions":[{"Height":[{"_":"752","$":{"Units":"hundredths-inches"}}],"Length":[{"_":"531","$":{"Units":"hundredths-inches"}}],"Weight":[{"_":"19","$":{"Units":"hundredths-pounds"}}],"Width":[{"_":"51","$":{"Units":"hundredths-inches"}}]}],"ItemPartNumber":["710425491016"],"Label":["2K"],"Languages":[{"Language":[{"Name":["English"],"Type":["Published"]},{"Name":["English"],"Type":["Original Language"]},{"Name":["English"],"Type":["Unknown"]}]}],"ListPrice":[{"Amount":["1999"],"CurrencyCode":["USD"],"FormattedPrice":["$19.99"]}],"Manufacturer":["2K"],"ManufacturerMinimumAge":[{"_":"96","$":{"Units":"months"}}],"Model":["49101"],"MPN":["49101"],"NumberOfItems":["1"],"OperatingSystem":["Xbox 360"],"PackageDimensions":[{"Height":[{"_":"50","$":{"Units":"hundredths-inches"}}],"Length":[{"_":"750","$":{"Units":"hundredths-inches"}}],"Weight":[{"_":"20","$":{"Units":"hundredths-pounds"}}],"Width":[{"_":"530","$":{"Units":"hundredths-inches"}}]}],"PackageQuantity":["1"],"PartNumber":["49101"],"Platform":["Xbox 360"],"ProductGroup":["Video Games"],"ProductTypeName":["CONSOLE_VIDEO_GAMES"],"PublicationDate":["2012-09-18"],"Publisher":["2K"],"RegionCode":["36222"],"ReleaseDate":["2012-09-18"],"Studio":["2K"],"Title":["Borderlands 2 - Xbox 360"],"TradeInValue":[{"Amount":["86"],"CurrencyCode":["USD"],"FormattedPrice":["$0.86"]}],"UPC":["021113169989"],"UPCList":[{"UPCListElement":["021113169989","100177413464","031111574449","163120331895","088021472273","745449851300","151903126141","710425491016"]}]}]
I can not find a way to retrieve the field title from the results!
Try this:
client.itemSearch({
keywords:'borderlands 2' ,
searchIndex: 'VideoGames',
responseGroup: 'ItemAttributes,Images'
}, function(err, results, response) {
if (err) {
console.log(err);
} else {
var amazon_product = JSON.parse(results[0].ItemAttributes);
console.log(amazon_product.Title);
}
});
I hope it helps :)

What is pass-reference-by-value?

I came across this yesterday and was wondering what the relationship of pass-by-value, pass-by-reference and pass-reference-by-value is. How do they all behave? Which programming language uses which?
Additionally:
I have also occasionally heard other terms like pass-by-pointer, pass-by-copy, pass-by-object...how do they all fit in?
It's a made-up term that refers to something that is actually pass-by-value, but where people always get confused because the only values in the language are references (pointers to things). Many modern languages work this way, including Java (for non-primitives), JavaScript, Python, Ruby, Scheme, Smalltalk, etc.
Beginners always ask questions on StackOverflow about why it's not pass-by-reference, because they don't understand what pass-by-reference really is. So to satisfy them some people make up a new term that they also don't understand, but makes it seem like they learned something, and which sounds vaguely like both pass-by-value and pass-by-reference.
Use of terminology varies widely between different language communities. For example, the Java community almost always describes this semantics as pass-by-value, while the Python and Ruby communities rarely describe it as pass-by-value, even though they are semantically identical.
Let's start with the basics. You probably already know this or at least think you do. Surprisingly many people are actually misusing these terms quite a bit (which certainly doesn't help with the confusion).
What is pass-by-value and pass-by-reference?
I am not going to tell you the answer just yet and explain it using examples instead. You will see why later.
Imagine you are trying to learn a programming language and a friend tells you that he knows a great book on the topic. You have two options:
borrow the book from him or
buy your own copy.
Case 1: If you borrow his book and you make notes in the margins (and add some bookmarks, references, drawings, ...) your friend can later benefit from those notes and bookmarks as well. Sharing is caring. The world is great!
Case 2: But if, on the other hand, your friend is a bit of a neat freak or holds his books very dear he might not appreciate your notes and even less the occasional coffee stain or how you fold the corners for bookmarking. You should really get your own book and everybody can happily treat his book the way he likes. The world is great!
These two cases are - you guessed it - just like pass-by-reference and pass-by-value.
The difference is whether the caller of the function can see changes that the callee makes.
Let's look at a code example.
Say you have the following code snippet:
function largePrint(text):
text = makeUpperCase(text)
print(text)
myText = "Hello"
largePrint(myText)
print(myText)
and you run it in a pass-by-value language the output will be
HELLO
Hello
while in a pass-by-reference language the output will be
HELLO
HELLO
If you pass-by-reference the variable "myText" gets changed by the function. If you just pass the value of the variable the function cannot change it.
If you need another example, this one using websites got quite a few upvotes.
Okay, now that we have the basics down let's move on to...
What is pass-reference-by-value?
Let's use our book example again. Assume that you have decided to borrow the book from your friend although you know how much he loves his books. Let us also assume that you then - being the total klutz that you are - spilled a gallon of coffee over it. Oh, my...
Not to worry, your brain starts up and realizes that you can just buy your friend a new book. It was in such good condition he won't even notice!
What does that story have to do with pass-reference-by-value?
Well, imagine on the other hand that your friend loved his books so muchs he doesn't want to lend it to you. He does however agree to bring the book and show it to you. The problem is, you can still spill a gallon of coffee over it but now you can't buy him a new book anymore. He would notice!
While this may sound dreadful (and probably has you sweating as if you had drunk that gallon of coffee) but it is actually quite common and a perfectly good way to share books call functions. It is sometimes called pass-reference-by-value.
How does the code example fare?
In a pass-reference-by-value function the output of our snippet is
HELLO
Hello
just like in the pass-by-value case. But if we change the code a little bit:
function largePrint(text):
text.toUpperCase()
print(text)
myText = "Hello"
largePrint(myText)
print(myText)
The output becomes:
HELLO
HELLO
just like in the pass-by-reference case. The reason is the same as in the book example: We can change the variable (or book) by calling text.toUpperCase() (or by spilling coffee on it). BUT we cannot change the object that the variable references anymore (text = makeUppercase(text) has no effect); in other words we cannot use a different book.
So summarizing we get:
pass-by-value:
You get your own book, your own copy of the variable and whatever you do with it the original owner will never know.
pass-by-reference:
You use the same book as your friend or the same variable as the caller. If you change the variable it changes outside your function.
pass-reference-by-value
You use the same book as your friend but he doesn't let it out of his sight. You can change the book but not EXchange it. In variable terms that means that you cannot change what the variable references but you can change that which the variable references.
So now when I show you the following Python code:
def appendSeven(list):
list.append(7)
def replaceWithNewList(list):
list = ["a", "b", "c"]
firstList = [1, 2, 3, 4, 5, 6]
print(firstList)
appendSeven(firstList)
print(firstList)
secondList = ["x", "y", "z"]
print(secondList)
replaceWithNewList(secondList)
print(secondList)
and tell you that the output is this
[1, 2, 3, 4, 5, 6]
[1, 2, 3, 4, 5, 6, 7]
['x', 'y', 'z']
['x', 'y', 'z']
what do you say?
Indeed, you are correct! Python uses pass-reference-by-value! (And so is Java and Ruby and Scheme and...)
Conclusion:
Was this extremely difficult? No.
Then why are so many people confused by it? (I'm not going to link to all the other questions about "How do I pass a variable by reference in Python" or "Is Java pass-by-reference?" and so on...)
Well, I see several reasons:
Saying pass-value-by-reference is really long (and it behaves almost the same as pass-by-reference anyway so get of my back and stop splitting hairs!)
There are nasty special cases that complicate the issue
like primitive types (which are often pass-by-value in pass-value-by-reference languages)
immutable types (which cannot be changed so passing them by-reference doesn't get you anywhere)
Some people just like to use different names:
pass-by-copy for pass-by-value because if you paid attention, you need your own book, so a copy has to be made. (But that is an implementation detail, other tricks like copy-on-write might be used to only sometimes make a copy.)
pass-by-reference for pass-reference-by-value because, after all, you did get a reference and that means you can change the object that the caller had (even if you cannot EXchange it)
pass-by-reference for pass-reference-by-value because almost no language actually has pass-by-reference (notable Exceptions (that I know): C++, C# but you have to use special syntax)
pass-by-sharing for pass-reference-by-value (remember our book example? that name is actually way better and everybody should use it!)
pass-by-object for pass-reference-by-value because allegedly Barbara Liskov first named it and used that, so that's its birthname)
pass-by-pointer for pass-by-reference because C doesn't have pass-by-reference and that is how it allows you to accomplish the same thing.
If there is anything you think I should add, please let me know. If I made a mistake, PLEASE LET ME KNOW!
For those of you who still haven't got enough: http://en.wikipedia.org/wiki/Evaluation_strategy

Banned words checking algo

I am building a text chat system. I want to add the ability to check for banned words/phrases.
The only technique I can think of, and can't believe it could possibly be the best approach is to do a FOR loop through all the words and search for matches in the text. This seems like it would be unbelievably slow once lots of words are added.
I'm using AS3, but an answer in most any language would probably be useful.
take care,
lee
use an AS3 dictionary or a dict in python and just check if the word is in the dict. there is no way I can see to not go over all the words.
Consider concatenating all the entries in your Dictionary into a single RegExp, with which you have to parse the text only once. I've done some testing, and it's going to be way faster than replacing word for word.
function censorWithDictionary ( dict:Dictionary, text:String ) : String {
var reg : String = "";
for (var key:Object in dict)
{
reg += reg=="" ? "" : "|"; // add an "or" for multiple search words
reg += "\\b"+dict[key]+"\\b"; // only whole words
}
var regExp : RegExp = new RegExp ( reg, "gi" );
return text.replace ( regExp, "----" );
}
I had a similar problem - we run a gaming site and wanted to introduce a chat system which was not manually moderated. We went the "banned word" route and it's working really well.
I just counted them and we now have a list of (just) 79 banned words which originated from something I found on-line to which we have added words over time when chat messages crept through.
The way we check things is that we concatenate an entire chat message by removing all spaces and none alpha characters and then search for banned words in what's left.
The key decisions we made are:
Don't tell people why you rejected their messages
Don't let people post chat until you trust them a bit (on our site they have
to have played 3 games)
5 "Bad" messages and we automatically block you
We email a report out daily with all the chat which got through which we scan through
We allow other users to complain about posted messages - if that happens the message is automatically removed so we can check it later.
1+3+5 Hardly ever happen now and it works wonderfully even though - sometimes messages like
"I wish it was hot!"
Are rejected (the clue is the "sh" part of wish and "it") but even that doesn't happen often.
This is more a comment than an answer, but comments are limited in length and there're big issues here.
I believe you are fundamentally asking the wrong question!
Certainly dictionaries and blacklist would highlight words or phrases that you want to ban but would that list be acceptable to users of your system? Would there be text that users of your system find offensive but you do not. Who decides?
For example, would people living here have trouble or indeed people living here. What if you supported this football/soccer team. This person probably never visits the UK.
Then you get into the issue of anagrams and slang. FCUK is a high street brand in the UK (and elsewhere I'm sure). And then there's pr0n (no link!) or NAMBLA.
The real question is - How do I stop people using the system from using language that is generally unacceptable? And that's more a design / social engineering problem than a programming problem. I don't think this site has word / phrase filtering and yet there's nothing here that would cause offense to anyone.
Here's an idea - let your users decide what is acceptable! Use a reputation based system. Allow users to vote up users who behave and vote down users that cause offense (with the option of allowing users to give feedback on the vote to give them a chance to mend their ways) and then have an option to filter out users with low / negative reputations.

How can I program a simple chat bot AI?

I want to build a bot that asks someone a few simple questions and branches based on the answer. I realize parsing meaning from the human responses will be challenging, but how do you setup the program to deal with the "state" of the conversation?
It will be a one-to-one conversation between a human and the bot.
You probably want to look into Markov Chains as the basics for the bot AI. I wrote something a long time ago (the code to which I'm not proud of at all, and needs some mods to run on Python > 1.5) that may be a useful starting place for you: http://sourceforge.net/projects/benzo/
EDIT: Here's a minimal example in Python of a Markov Chain that accepts input from stdin and outputs text based on the probabilities of words succeeding one another in the input. It's optimized for IRC-style chat logs, but running any decent-sized text through it should demonstrate the concepts:
import random, sys
NONWORD = "\n"
STARTKEY = NONWORD, NONWORD
MAXGEN=1000
class MarkovChainer(object):
def __init__(self):
self.state = dict()
def input(self, input):
word1, word2 = STARTKEY
for word3 in input.split():
self.state.setdefault((word1, word2), list()).append(word3)
word1, word2 = word2, word3
self.state.setdefault((word1, word2), list()).append(NONWORD)
def output(self):
output = list()
word1, word2 = STARTKEY
for i in range(MAXGEN):
word3 = random.choice(self.state[(word1,word2)])
if word3 == NONWORD: break
output.append(word3)
word1, word2 = word2, word3
return " ".join(output)
if __name__ == "__main__":
c = MarkovChainer()
c.input(sys.stdin.read())
print c.output()
It's pretty easy from here to plug in persistence and an IRC library and have the basis of the type of bot you're talking about.
Folks have mentioned already that statefulness isn't a big component of typical chatbots:
a pure Markov implementations may express a very loose sort of state if it is growing its lexicon and table in real time—earlier utterances by the human interlocutor may get regurgitated by chance later in the conversation—but the Markov model doesn't have any inherent mechanism for selecting or producing such responses.
a parsing-based bot (e.g. ELIZA) generally attempts to respond to (some of the) semantic content of the most recent input from the user without significant regard for prior exchanges.
That said, you certainly can add some amount of state to a chatbot, regardless of the input-parsing and statement-synthesis model you're using. How to do that depends a lot on what you want to accomplish with your statefulness, and that's not really clear from your question. A couple general ideas, however:
Create a keyword stack. As your human offers input, parse out keywords from their statements/questions and throw those keywords onto a stack of some sort. When your chatbot fails to come up with something compelling to respond to in the most recent input—or, perhaps, just at random, to mix things up—go back to your stack, grab a previous keyword, and use that to seed your next synthesis. For bonus points, have the bot explicitly acknowledge that it's going back to a previous subject, e.g. "Wait, HUMAN, earlier you mentioned foo. [Sentence seeded by foo]".
Build RPG-like dialogue logic into the bot. As your parsing human input, toggle flags for specific conversational prompts or content from the user and conditionally alter what the chatbot can talk about, or how it communicates. For example, a chatbot bristling (or scolding, or laughing) at foul language is fairly common; a chatbot that will get het up, and conditionally remain so until apologized to, would be an interesting stateful variation on this. Switch output to ALL CAPS, throw in confrontational rhetoric or demands or sobbing, etc.
Can you clarify a little what you want the state to help you accomplish?
Imagine a neural network with parsing capabilities in each node or neuron. Depending on rules and parsing results, neurons fire. If certain neurons fire, you get a good idea about topic and semantic of the question and therefore can give a good answer.
Memory is done by keeping topics talked about in a session, adding to the firing for the next question, and therefore guiding the selection process of possible answers at the end.
Keep your rules and patterns in a knowledge base, but compile them into memory at start time, with a neuron per rule. You can engineer synapses using something like listeners or event functions.
I think you can look at the code for Kooky, and IIRC it also uses Markov Chains.
Also check out the kooky quotes, they were featured on Coding Horror not long ago and some are hilarious.
I think to start this project, it would be good to have a database with questions (organized as a tree. In every node one or more questions).
These questions sould be answered with "yes " or "no".
If the bot starts to question, it can start with any question from yuor database of questions marked as a start-question. The answer is the way to the next node in the tree.
Edit: Here is a somple one written in ruby you can start with: rubyBOT
naive chatbot program. No parsing, no cleverness, just a training file and output.
It first trains itself on a text and then later uses the data from that training to generate responses to the interlocutor’s input. The training process creates a dictionary where each key is a word and the value is a list of all the words that follow that word sequentially anywhere in the training text. If a word features more than once in this list then that reflects and it is more likely to be chosen by the bot, no need for probabilistic stuff just do it with a list.
The bot chooses a random word from your input and generates a response by choosing another random word that has been seen to be a successor to its held word. It then repeats the process by finding a successor to that word in turn and carrying on iteratively until it thinks it’s said enough. It reaches that conclusion by stopping at a word that was prior to a punctuation mark in the training text. It then returns to input mode again to let you respond, and so on.
It isn’t very realistic but I hereby challenge anyone to do better in 71 lines of code !! This is a great challenge for any budding Pythonists, and I just wish I could open the challenge to a wider audience than the small number of visitors I get to this blog. To code a bot that is always guaranteed to be grammatical must surely be closer to several hundred lines, I simplified hugely by just trying to think of the simplest rule to give the computer a mere stab at having something to say.
Its responses are rather impressionistic to say the least ! Also you have to put what you say in single quotes.
I used War and Peace for my “corpus” which took a couple of hours for the training run, use a shorter file if you are impatient…
here is the trainer
#lukebot-trainer.py
import pickle
b=open('war&peace.txt')
text=[]
for line in b:
for word in line.split():
text.append (word)
b.close()
textset=list(set(text))
follow={}
for l in range(len(textset)):
working=[]
check=textset[l]
for w in range(len(text)-1):
if check==text[w] and text[w][-1] not in '(),.?!':
working.append(str(text[w+1]))
follow[check]=working
a=open('lexicon-luke','wb')
pickle.dump(follow,a,2)
a.close()
here is the bot
#lukebot.py
import pickle,random
a=open('lexicon-luke','rb')
successorlist=pickle.load(a)
a.close()
def nextword(a):
if a in successorlist:
return random.choice(successorlist[a])
else:
return 'the'
speech=''
while speech!='quit':
speech=raw_input('>')
s=random.choice(speech.split())
response=''
while True:
neword=nextword(s)
response+=' '+neword
s=neword
if neword[-1] in ',?!.':
break
print response
You tend to get an uncanny feeling when it says something that seems partially to make sense.
I would suggest looking at Bayesian probabilities. Then just monitor the chat room for a period of time to create your probability tree.
I'm not sure this is what you're looking for, but there's an old program called ELIZA which could hold a conversation by taking what you said and spitting it back at you after performing some simple textual transformations.
If I remember correctly, many people were convinced that they were "talking" to a real person and had long elaborate conversations with it.
If you're just dabbling, I believe Pidgin allows you to script chat style behavior. Part of the framework probably tacks the state of who sent the message when, and you'd want to keep a log of your bot's internal state for each of the last N messages. Future state decisions could be hardcoded based on inspection of previous states and the content of the most recent few messages. Or you could do something like the Markov chains discussed and use it both for parsing and generating.
If you do not require a learning bot, using AIML (http://www.aiml.net/) will most likely produce the result you want, at least with respect to the bot parsing input and answering based on it.
You would reuse or create "brains" made of XML (in the AIML-format) and parse/run them in a program (parser). There are parsers made in several different languages to choose from, and as far as I can tell the code seems to be open source in most cases.
You can use "ChatterBot", and host it locally using - 'flask-chatterbot-master"
Links:
[ChatterBot Installation]
https://chatterbot.readthedocs.io/en/stable/setup.html
[Host Locally using - flask-chatterbot-master]: https://github.com/chamkank/flask-chatterbot
Cheers,
Ratnakar