How do I get the spoken text from the hotword.py code & do my own actions on the recognised text rather than Google going off and reacting to the text?
I've installed GA on the Pi3 & after some initial issues with usb mic/analogue audio settings and certain Python files missing this got me going:
When installing Google Assistant, I an error "...googlesamples.assistant' is a package and cannot be directly executed..."
I then followed the Google Next steps : https://developers.google.com/assistant/sdk/prototype/getting-started-pi-python/run-sample and created a new project "myga/" with a hotword.py file that contains:
def process_event(event):
"""Pretty prints events.
Prints all events that occur with two spaces between each new
conversation and a single space between turns of a conversation.
Args:
event(event.Event): The current event to process.
"""
if event.type == EventType.ON_CONVERSATION_TURN_STARTED:
print()
#GPIO.output(25,True) see https://stackoverflow.com/questions/44219740/how-can-i-get-an-led-to-light-on-google-assistant-listening
if event.type == EventType.ON_RECOGNIZING_SPEECH_FINISHED:
print("got some work to do here with the phrase or text spoken!")
print(event)
if (event.type == EventType.ON_CONVERSATION_TURN_FINISHED and
event.args and not event.args['with_follow_on_turn']):
print()
#GPIO.output(25,False) or also see https://blog.arevindh.com/2017/05/20/voice-activated-google-assistant-on-raspberry-pi-with-visual-feedback/
I'd like code to react to the ON_RECOGNIZING_SPEECH_FINISHED event I think and at either do my own action by matching simple requests or if the phrase is not in my list then let Google handle it. How do I do that?
Eventually I'd be asking "OK Google, turn BBC1 on" or "OK Google, play my playlist" or "OK Google, show traffic" and hotword.py would run other applications to do those tasks.
Thanks, Steve
See the documentation here for all available methods -
https://developers.google.com/assistant/sdk/reference/library/python/
You can use the stop_conversation() method to stop Google Assistant handling that request and act on your own.
Here's what you need to do at a high level -
Build your own dictionary of commands that you'd like to handle -
"turn BBC1 on", "play my playlist" etc.
On EventType.ON_RECOGNIZING_SPEECH_FINISHED event check if the
recognized command exists in your dictionary.
If the recognized command exists in your dictionary call the assistant.stop_conversation() method and handle the command on your own. If not do nothing (let google handle it)
pseudo code -
local_commands = ['turnBBCOn', 'playLocalPlaylist']
function turnBBCOn() :
#handle locally
function playLocalPlaylist() :
#handle locally
def process_event(event):
if event.type == EventType.ON_CONVERSATION_TURN_STARTED:
print()
if event.type == EventType.ON_RECOGNIZING_SPEECH_FINISHED:
print(event.args['text'])
if event.args['text'] in local_commands:
assistant.stop_conversation()
if(event.args['text']='turn BBC1 on')
turnBBCOn()
elif(event.args['text']='play my playlist')
playLocalPlaylist()
if (event.type == EventType.ON_CONVERSATION_TURN_FINISHED and
event.args and not event.args['with_follow_on_turn']):
print()
I have recently integrated google assistant SDK with Raspberry Pi 3. I have taken reference from below git repository and created action.py and actionbase.py classes can handle my custom command. I found it very clean and flexible way to create your own custom commands.
You can register your custom command in action.py file like below
actor = actionbase.Actor()
actor.add_keyword(
_('ip address'), SpeakShellCommandOutput(
say, "ip -4 route get 1 | head -1 | cut -d' ' -f8",
_('I do not have an ip address assigned to me.')))
return actor
action.py
Write your custom code in action.py
"""Speaks out the output of a shell command."""
def __init__(self, say, shell_command, failure_text):
self.say = say
self.shell_command = shell_command
self.failure_text = failure_text
def run(self, voice_command):
output = subprocess.check_output(self.shell_command, shell=True).strip()
if output:
self.say(output.decode('utf-8'))
elif self.failure_text:
self.say(self.failure_text)
You can full source code here. https://github.com/aycgit/google-assistant-hotword
The text is contained within the event arguments. By calling event.args you can make use of the text. Here is an example.
https://github.com/shivasiddharth/GassistPi/blob/master/src/main.py
Related
I will provide the code for a portion. It doesn't continue after using commands within a while block. I think it is the code block text = get_audio().lower() that is giving me the error.
def process_text():
pass
if __name__ == "__main__":
assistant_speaks('''Hello, I am a Virtual Assistant.
I am here to make your life easier. You can command me to perform
various tasks such as calculating sums or opening applications.
Please tell me who you are.''')
name = 'are'
name = get_audio()
assistant_speaks("Hello, " + name + '.')
while 1:
assistant_speaks("What can i do for you?" + name + '.')
text = get_audio().lower()
if text == 0:
continue
if "exit" in str(text) or "bye" in str(text) or "sleep" in str(text):
assistant_speaks("Ok bye, " + name + '.')
break
# calling process text to process the query
process_text()
def process_text():
try:
if 'search' in input or 'play' in input:
# a basic web crawler using selenium
search_web(input)
return
elif "who are you" in input or "define yourself" in input:
speak = '''Hello, I am a virtual assistant.
I am here to make your life easier. You can command me to perform
various tasks such as calculating sums or opening applications.'''
assistant_speaks(speak)
return
Basically, it doesn't answer any questions when I ask the elifs. Past the try:
Does anyone have any ideas?
There are at least three mistakes you should correct in order to get the behavior you want from your program:
Replace input with text inside the try block. In Python, input is a built-in function, not a string. You already assigned the user's answer to the variable text, so why not use that one? :)
Define your process_text function before you call it. Practically, you should move the line where you call process_text() somewhere after the definition of that function.
There are also a couple of indentation errors, and the most important happens when you call the process_text() function: you should un-indent that line back one level, otherwise it will be executed inside the if "exit" in ... block.
You should also define process_text before the while loop, because you just need to define it once.
I'm trying to add some lua functionality to my existing conky setup so that repetitive "code" in my conky text can be cleaned up. For example, I have information for each mounted FS, each core, etc. where each row displayed in my panel differs ONLY by one parameter.
My first skeletal, attempt at using lua functions for this seems to run but displays nothing in my panel. I've only found very simple examples to base this on, so I may have made a simple error, but I don't even know how to diagnose it. My code here is modeled after what I HAVE been able to find regarding writing functions, such as this How to implement a basic Lua function in Conky? , but that's about all the depth I've found on the topic except for drawing and cairo examples.
Here's the code added to my conky config, as well as the contents of my functions.lua file
conky.config = {
...
lua_load = '/home/conky-manager/MyConky/functions.lua',
};
conky.text = [[
...
${voffset 5}${lua conky_test 'test'}
...
]]
file - functions.lua
function conky_test(parm1)
return 'result text'
end
What I would expect is to see is "result text" displayed in my panel at the location where that function call appears, but nothing shows.
Is there a log created by conky as it runs, or a way to provide some debug output? Even if I'd made a simple error here, I'd still like to have the ability to diagnose things as my code gets more complex.
Success!
After cobbling info from several articles together, I figured out my basic flaws -
1. Missing a 'conky_main' function,
2. Missing a 'lua_draw_hook_post' to invoke it, and
3. Realizing that if I invoke conky from a terminal, print statements in lua would appear there.
So, for anyone who sees this question and has the same issues, here's the corrected code.
conky.config = {
...
lua_load = '/home/conky-manager/MyConky/functions.lua',
lua_draw_hook_post = "main",
};
conky.text = [[
...
${lua conky_test 'test'}
...
]]
and the proper basics in my functions.lua file
function conky_test(parm1)
return 'result text'
end
function conky_main()
if conky_window == nil then
return
end
end
A few notes:
I still haven't determined if using 'lua_draw_hook_pre' instead of 'lua_draw_hook_post' makes any difference, but it doesn't seem to in this example.
Also, some examples showed actually calling this 'test' function instead of writing a 'main', but the 'main' seemed to have value in checking to see if conky_window existed.
Some examples seemed to state that naming functions with the prefix 'conky_' was required, but then showed examples of calling those functions without the prefix, so I assume the prefix is inferred during the call.
a major note: you should run conky from the directory containing the lua scripts.
I have a test where I need to write the results into a database. I want to set up the connection to database (using username, password, database, host) via startSuite function in listener (which will run at the beginning of all testcases) and close it in endSuite. My question is, how do I pass the connection (or cursors) back to Robot Framework code to use in testcases.
Currently I am doing this:
*** Test Cases ***
RecordinTestflow
Setup1
${return} = Record Start in Testflow ${data}
where Setup1 is a python function which will setup the connection and RecordStartinTestFlow will use that connection. I want to move the Setup1 to a listener python script.
Thank you.
An external listener can't send information to a test case. However, if you use a keyword library as a listener, it can. The downside is that you have to import the listener in the test suite, rather than specify it on the command line.
The robot framework user guide has a section title Test libraries as listeners which describes how to do it.
Here's a contrived example showing how a listener method can set a suite variable which the test case can then use.
First, the listener:
from robot.libraries.BuiltIn import BuiltIn
class ListenerExample(object):
ROBOT_LISTENER_API_VERSION = 2
ROBOT_LIBRARY_SCOPE = "GLOBAL"
def __init__(self):
self.ROBOT_LIBRARY_LISTENER = self
def _start_suite(self, name, attrs):
message = "hello, world"
BuiltIn().set_suite_variable("${from listener}", message)
Next, a simple test case that shows how the variable gets set as soon as the suite starts. Notice that the test itself doesn't define ${from listener}. Instead, it gets defined as soon as the listener method is called.
*** Settings ***
| Library | ListenerExample.py
*** Test Cases ***
| Example of getting data from a listener
| | should be equal | ${from listener} | hello, world
In your case, of course, you would change message to be your database cursor or whatever else you want it to be.
Of course, you can also put keywords in this library that you can use as well.
i'm using CSS3 accordion effect, and i want to detect if a hacker will
make a script to make a parallel request; ie:
i've a login form and a registration form in the same page, but only
one is visible because there is a CSS3: to access the page, the user
agent must be HTML5 compatible.
the tip i use is:
class Register(tornado.web.RequestHandler):
def post(self):
tt = self.get_argument("_xsrf") + str(time.time())
rtime = float(tt.replace(self.get_argument("_xsrf"), ""))
print rtime
class LoginHandler(BaseHandler):
def post(self):
tt = self.get_argument("_xsrf") + str(time.time())
ltime = float(tt.replace(self.get_argument("_xsrf"), ""))
print ltime
i've used the xsrf variable because it's unique for every user, to
avoid making the server think that the request is coming from the same
machine.
now what i want: how to make the difference between time values:
abs(ltime - rtime) ; mean, how do i access to rtime outside the class,
i just know how to access the value outside the method, i want to make
this operation to detect if the value is small, then the user is using
a script to make a parallel request to kill the server!
in other words (for general python users)
if i have:
class Product:
def info(self):
self.price = 1000
def show(self):
print self.price
>>> car = Product()
>>> car.info()
>>> car.show()
1000
but what if i've another
class User:
pass
then how do make a method that prints me the self.price, i've tried
inheritance, but got error: AttributeError: User instance has no
attribute 'price', so only methods are passed, not attributs?
It sounds like you need to understand Model objects and patterns that use persistant storage of data. tornado.web.RequestHandler and any object that you subclass from it only exists for the duration of your request. From when the URL is received on the server to when data is sent back to the browser via a self.write() or self.finish().
I would recommend you look at some of the Django or Flask tutorials for some basic ideas of how to build a MVC application in Python (There is no Tornado Tutorials that cover this that I know of).
I had made a daemon that used a very primitive form of ipc (telnet and send a String that had certain words in a certain order). I snapped out of it and am now using JSON to pass messages to a Yesod server. However, there were some things I really liked about my design, and I'm not sure what my choices are now.
Here's what I was doing:
buildManager :: Phase -> IO ()
buildManager phase = do
let buildSeq = findSeq phase
jid = JobID $ pack "8"
config = MkConfig $ Just jid
flip C.catch exceptionHandler $
runReaderT (sequence_ $ buildSeq <*> stages) config
-- ^^ I would really like to keep the above line of code, or something like it.
return ()
each function in buildSeq looked like this
foo :: Stage -> ReaderT Config IO ()
data Config = MkConfig (Either JobID Product) BaseDir JobMap
JobMap is a TMVar Map that tracks information about current jobs.
so now, what I have are Handlers, that all look like this
foo :: Handler RepJson
foo represents a command for my daemon, each handler may have to process a different JSON object.
What I would like to do is send one JSON object that represents success, and another JSON object that espresses information about some exception.
I would like foos helper function to be able to return an Either, but I'm not sure how I get that, plus the ability to terminate evaluation of my list of actions, buildSeq.
Here's the only choice I see
1) make sure exceptionHandler is in Handler. Put JobMap in the App record. Using getYesod alter the appropriate value in JobMap indicating details about the exception,
which can then be accessed by foo
Is there a better way?
What are my other choices?
Edit: For clarity, I will explain the role ofHandler RepJson. The server needs some way to accept commands such as build stop report. The client needs some way of knowing the results of these commands. I have chosen JSON as the medium with which the server and client communicate with each other. I'm using the Handler type just to manage the JSON in/out and nothing more.
Philosophically speaking, in the Haskell/Yesod world you want to pass the values forward, rather than return them backwards. So instead of having the handlers return a value, have them call forwards to the next step in the process, which may be to generate an exception.
Remember that you can bundle any amount of future actions into a single object, so you can pass a continuation object to your handlers and foos that basically tells them, "After you are done, run this blob of code." That way they can be void and return nothing.