core.async reconnecting websocket - clojurescript

core.async noob here, but trying to learn by building an automatically reconnecting websocket. The idea is that the socket is abstracted away so that anyone using it does not need to worry about whether or not it's connected and can just put messages on a channel. If the socket is connected, the messages are read from the channel immediately and sent. If the socket is not currently connected, it repeatedly tries to reconnect and waits to send all pending messages once a connection has been established.
This is what I have so far:
(ns frontend.socket
(:require-macros [cljs.core.async.macros :as asyncm :refer [go]])
(:require [cljs.core.async :as async :refer [<! >! chan timeout]]))
(def endpoint "ws://localhost:8080")
(def socket (atom nil))
(def open (chan))
(def in (chan))
(def out (chan))
(def error (chan))
(def close (chan))
(defn open? []
(= (.-readyState #socket) (.-OPEN #socket)))
(defn event>chan
"Given a core.async channel, returns a
function which takes an event and feeds
it into the formerly given channel."
[channel]
(fn [e]
(go (>! channel e))))
(defn connect
"Opens a websocket, stores it in the socket
atom, and feeds the socket's events into
the corresponding channels."
[]
(let [s (js/WebSocket. endpoint)]
(reset! socket s)
(set! s.onopen (event>chan open))
(set! s.onmessage (event>chan in))
(set! s.onerror (event>chan error))
(set! s.onclose (event>chan close))))
(defn init
"This is the entry point from outside this
namespace. Eagerly connects and then, if
ever disconnected, attempts to reconnect
every second. Also takes messages from
the out channel and sends them through
the socket."
[]
(go
(while true
(connect)
(<! close)
(<! (timeout 1000))))
(go
(while true
(let [m (<! out)]
(when (or (open?) (<! open))
(.send #socket m))))))
The reconnect logic seems to work fine, but I'm running into an issue trying to send messages after a socket has closed. In the last few lines, I check that before sending a message, the socket is open or wait for it to open. The problem is that if no message was sent while a socket was previously open, then closed, and then a message is sent, there is still something sitting on the open channel, and so the message will get sent regardless. So the open channel can get "stale", as it were.
I thought about taking from the open channel whenever the socket closes, but that just reverses the problem: then the open channel may miss values when I do need them, and messages are not sent upon reconnect.
What am I doing wrong here?

This was the wrong approach because channels will wait to be consumed. I needed to use pub/sub instead: https://github.com/clojure/core.async/wiki/Pub-Sub/549da1843c7bbe6009e9904eed49f91000d8ce2c

Related

Why my pub sub message acknowledged? -> Data loss

I have a cloud function which is called Pub/Sub. It should ACK the message only when it has been correctly processed.
I see in my logs a memory failure, then a message 'Finished with status: ok'.
And the message is acknowledged and removed from my Pub / Sub topic!
Reproducer:
import base64
import requests
def hello_pubsub(event, context):
""" Triggered from a message on a Cloud Pub/Sub topic.
Args:
event (dict): Event payload.
context (google.cloud.functions.Context): Metadata for the event.
"""
pubsub_message = base64.b64decode(event['data']).decode('utf-8')
# let's say that important stuff is done at the line below
# which should be retried in case of failure
r = requests.get('https://zoom.us/client/latest/zoomusInstallerFull.pkg')
print(pubsub_message)
If the Google Cloud Function is going to be executed, it follows that the subject has already received the ack message.
The messages will receive an ACK if your function successfully completes by returning a resolved promise. PubSub will retry the message if the function throws an error or returns a denied promise.
Therefore, to prevent looping, you must build a retry method that will try a predetermined number of times and then quit. This is necessary if your Cloud Function was unable to process the received message.
Now, in accordance with the previous comment, try allowing retry on failure when an executionID is connected to the request that resulted in error. When a retriable exception is raised, this enables function execution to be retried.
Reviewing for the exact error message that is showing in Cloud Functions which is “Function invocation was interrupted. Error: function terminated”, tt would be advised to check the logs for termination-related information.
Here is further troubleshooting information from the Logging section. Setting up logs to aid in problem-solving can lead to more issues.

When do GCP cloud functions acknowledge pub/sub messages?

I have a cloud function that gets triggered from a pub/sub message. This function never explicitly acknowledges the message in the source code.
So when does this function acknowledge the pub/sub message if the acknowledgement never happens in the source code?
Update: when a function crashes, I understand that a message acknowledgement shouldn't occur and yet a new function invocation for that message never appears in the logs
Reproducible Example
Create a pubsub topic called test_topic
Create a cloud function called test_function with trigger test_topic. Give it all the default settings including NOT retrying on failure. In the code itself, set the language to python3.7 with entry point of hello_pubsub and the following code:
import base64
def hello_pubsub(event, context):
pubsub_message = base64.b64decode(event['data']).decode('utf-8')
print(pubsub_message)
raise RuntimeError('error in function')
The requirements.txt remains blank
Go into test_topic and publish a message with go as the text.
There will be an error in the test_function logs. However there will only be one function invocation with the error and this will remain the case even after a few days or so.
If the function finish gracefully, the message is acknowledge. If the function exits in error, the message is NACK.
EDIT 1
I have tested with a Go background function. You need to deploy your cloud function with the parameter --retry to allow the messages in error to be retried. Else, the messages aren't retried.
In Go, here the cases where retried are performed:
Return an Error (equivalent to exception in Java or Python), status "error" in the logs
Perform a log.Fatal() (exit the function (function crash) with a specific log) status "connection error" in the logs
Perform an explicit exit, status "connection error" in the logs
Here the code (if interested)
type PubSubMessage struct {
Data []byte `json:"data"`
}
func PubsubError(ctx context.Context, m PubSubMessage) error {
switch string(m.Data) {
case "ok":
return nil
case "error":
return errors.New("it's an error")
case "fatal":
log.Fatal("crash")
case "exit":
os.Exit(1)
}
return nil
}
And how i deployed my Cloud Functions
gcloud beta functions deploy --runtime=go113 --trigger-topic=test-topic \
--source=function --entry-point=PubsubError --region=us-central1 \
--retry pubsuberror
Based on this description:
Google Cloud Pub/Sub Triggers
Cloud Functions acks the Pub/Sub message internally upon successful function invocation.
I do understand that documentation quotation as the acknowledgement happens only after the execution of code is finished without any (uncatched) errors.
At the same time, while the execution might still be 'in progress', the Pub/Sub service may make a decision to trigger another cloud function (instance) from the same Pub/Sub message.
Some additional details are in this Issue Tracker dicussion:
Cloud Function explicit acknowledgement of a pubsub message
From my point of view, independently from 'successful' or 'not successful' the invocation happened, the cloud function is to be developed in an idempopent way, taking into account 'at least once delivery' paradigm of the Pub/Sub service. In other words the cloud function is to be developed in a such a way, that multiple invocations from one message are handled correctly.

URI is accessible and downloadable from browser, but slurp/reader returns an empty vector

I am new to Clojure, and I am trying to read a set of data from Yahoo's historical finance API. I've used slurp and reader on other URIs which seem to be fine, but I seem to be unable to read this specific one.
The problem URI: http://ichart.finance.yahoo.com/table.csv?s=WU&a=11&b=15&c=2016&d=11&e=19&f=2016&g=d&ignore=.csv
Accessing this from the browser results in a CSV file with data in it, but slurp/reader returns an empty vector. I've tried:
(def uri-string
(str "http://ichart.finance.yahoo.com/table.csvs=WU&a=11&b=15&c=2016&d=11&e=19&f=2016&g=d&ignore=.csv"))
(slurp uri-string)
and:
(with-open [rdr (clojure.java.io/reader "http://ichart.finance.yahoo.com/table.csv?s=WU&a=11&b=15&c=2016&d=11&e=19&f=2016&g=d&ignore=.csv")] (reduce conj [] (line-seq rdr)))
but neither work. Anyone have any ideas on how to proceed, or what the problem is here? Thanks in advance!

Google Chrome and Chromium, "Aw, Snap" under all conditions.... how to isolate reason

I have been using Google Chrome on my Slackware unix box for about 6 months now. So I know the program does work on Slackware. But -- I recently had to recompile and upgrade my linux kernel from 3.17.8 to 3.18.9 in order to upgrade the system's ATI R9 290 graphics card behavior for games, and I also did it in hopes of eliminating a google-chrome ERROR start up message which says "ERROR:sandbox_linux.cc(325)] InitializeSandbox() called with multiple threads in process gpu-process" , BUT shortly afterward I started getting the Aw, Snap! blue screen of death and can no longer web browse.
I tried all the obvious repair tricks, deleting the google ~/.cache/... and ~/.config/..., reinstalling chrome (and got & compiled the latest releases google-chrome-41.0.2272.101-x86_64-1, chromium-41.0.2272.101-x86_64-1alien, ), to the extreme attempt in desperation of completely re-installing Slackware linux from scratch on a new hard drive assuming I must have damaged a library or something somewhere and even recompiling the kernel both with and without seccomp trusted byte code, and switching back and forth between mozilla-nss-3.15.2-x86_64-2, and seamonkey-solibs-2.21-x86_64-1 to see if either library set caused a problem...
And just to be safe, I installed all the 32 bit multilib compatability libraries for every 64 bit package I installed, and added the following non-standard libraries to the system which some websites suggested might have been needed in the past for Chrome, even if they aren't anymore...
ORBit2-2.14.19-x86_64-1_SBo
GConf-3.2.6-x86_64-1
pciutils-3.2.0-x86_64-2 # To fix slackware libpci.a not being a dynamic lib.
libgnome-keyring-3.8.0-x86_64-1
gnome-keyring-3.8.2-x86_64-1
freealut-1.1.0-x86_64-1ponce
freeglut-2.8.0-x86_64-1
OpenAL-1.16.0-x86_64-1_SBo
And for fonts, I have true microsoft windows Times, and Arial, and also dejavu-fonts-ttf-2.34-noarch-1, and fslsfonts-1.0.4-x86_64-1, which have always been sufficient in the past, but to be safe I added liberation-fonts-ttf-1.07.2-noarch-1 recently; but it didn't help.
The present ldd of chromium shows the following libraries, all of them being stock for slackware64 14.1 except the ones listed above; As you can see, there are NO missing libraries.
linux-vdso.so.1 (0x00007fff60fda000)
librt.so.1 => /lib64/librt.so.1 (0x00007f8a60296000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f8a60091000)
libgobject-2.0.so.0 => /usr/lib64/libgobject-2.0.so.0(0x00007f8a5fe42000)
libglib-2.0.so.0 => /usr/lib64/libglib-2.0.so.0 (0x00007f8a5fb18000)
libnss3.so => /usr/lib64/libnss3.so (0x00007f8a5f7db000)
libsmime3.so => /usr/lib64/libsmime3.so (0x00007f8a5f5ae000)
libnssutil3.so => /usr/lib64/libnssutil3.so (0x00007f8a5f383000)
libplc4.so => /usr/lib64/libplc4.so (0x00007f8a5f17e000)
libnspr4.so => /usr/lib64/libnspr4.so (0x00007f8a5ef41000)
libgio-2.0.so.0 => /usr/lib64/libgio-2.0.so.0 (0x00007f8a5ebe8000)
libfontconfig.so.1 => /usr/lib64/libfontconfig.so.1 (0x00007f8a5e9ad000)
libfreetype.so.6 => /usr/lib64/libfreetype.so.6 (0x00007f8a5e71c000)
libpangocairo-1.0.so.0 => /usr/lib64/libpangocairo-1.0.so.0 (0x00007f8a5e510000)
libcairo.so.2 => /usr/lib64/libcairo.so.2 (0x00007f8a5e21b000)
libpango-1.0.so.0 => /usr/lib64/libpango-1.0.so.0 (0x00007f8a5dfd1000)
libX11.so.6 => /usr/lib64/libX11.so.6 (0x00007f8a5dc97000)
libXi.so.6 => /usr/lib64/libXi.so.6 (0x00007f8a5da87000)
libXcursor.so.1 => /usr/lib64/libXcursor.so.1 (0x00007f8a5d87d000)
libXext.so.6 => /usr/lib64/libXext.so.6 (0x00007f8a5d66c000)
libXfixes.so.3 => /usr/lib64/libXfixes.so.3 (0x00007f8a5d466000)
libXrender.so.1 => /usr/lib64/libXrender.so.1 (0x00007f8a5d25d000)
libXcomposite.so.1 => /usr/lib64/libXcomposite.so.1 (0x00007f8a5d05b000)
libasound.so.2 => /usr/lib64/libasound.so.2 (0x00007f8a5cd65000)
libXdamage.so.1 => /usr/lib64/libXdamage.so.1 (0x00007f8a5cb63000)
libXtst.so.6 => /usr/lib64/libXtst.so.6 (0x00007f8a5c95e000)
libXrandr.so.2 => /usr/lib64/libXrandr.so.2 (0x00007f8a5c754000)
libexpat.so.1 => /usr/lib64/libexpat.so.1 (0x00007f8a5c52b000)
libcups.so.2 => /usr/lib64/libcups.so.2 (0x00007f8a5c2e0000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f8a5c0c2000)
libdbus-1.so.3 => /usr/lib64/libdbus-1.so.3 (0x00007f8a5be7c000)
libgtk-x11-2.0.so.0 => /usr/lib64/libgtk-x11-2.0.so.0 (0x00007f8a5b848000)
libgdk-x11-2.0.so.0 => /usr/lib64/libgdk-x11-2.0.so.0 (0x00007f8a5b595000)
libgdk_pixbuf-2.0.so.0 => /usr/lib64/libgdk_pixbuf-2.0.so.0 (0x00007f8a5b375000)
libXss.so.1 => /usr/lib64/libXss.so.1 (0x00007f8a5b172000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f8a5ae6f000)
libm.so.6 => /lib64/libm.so.6 (0x00007f8a5ab6d000)
libgcc_s.so.1 => /usr/lib64/libgcc_s.so.1 (0x00007f8a5a957000)
libc.so.6 => /lib64/libc.so.6 (0x00007f8a5a58d000)
/lib64/ld-linux-x86-64.so.2 (0x00007f8a679db000)
libffi.so.6 => /usr/lib64/libffi.so.6 (0x00007f8a5a385000)
libplds4.so => /usr/lib64/libplds4.so (0x00007f8a5a182000)
libz.so.1 => /lib64/libz.so.1 (0x00007f8a59f6c000)
libgmodule-2.0.so.0 => /usr/lib64/libgmodule-2.0.so.0 (0x00007f8a59d69000)
libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f8a59b4e000)
libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f8a5993e000)
libpng14.so.14 => /usr/lib64/libpng14.so.14 (0x00007f8a59719000)
libpixman-1.so.0 => /usr/lib64/../lib64/libpixman-1.so.0 (0x00007f8a59470000)
libxcb-shm.so.0 => /usr/lib64/../lib64/libxcb-shm.so.0 (0x00007f8a5926e000)
libX11-xcb.so.1 => /usr/lib64/../lib64/libX11-xcb.so.1 (0x00007f8a5906d000)
libxcb-render.so.0 => /usr/lib64/../lib64/libxcb-render.so.0 (0x00007f8a58e63000)
libxcb.so.1 => /usr/lib64/../lib64/libxcb.so.1 (0x00007f8a58c46000)
libXau.so.6 => /usr/lib64/../lib64/libXau.so.6 (0x00007f8a58a43000)
libXdmcp.so.6 => /usr/lib64/../lib64/libXdmcp.so.6 (0x00007f8a5883d000)
libpangoft2-1.0.so.0 => /usr/lib64/../lib64/libpangoft2-1.0.so.0 (0x00007f8a58629000)
libgthread-2.0.so.0 => /usr/lib64/../lib64/libgthread-2.0.so.0 (0x00007f8a58428000)
libharfbuzz.so.0 => /usr/lib64/../lib64/libharfbuzz.so.0 (0x00007f8a58196000)
libicule.so.51 => /usr/lib64/../lib64/libicule.so.51 (0x00007f8a57f40000)
libicuuc.so.51 => /usr/lib64/../lib64/libicuuc.so.51 (0x00007f8a57bd6000)
libicudata.so.51 => /usr/lib64/../lib64/libicudata.so.51 (0x00007f8a5648b000)
libssl.so.1 => /lib64/libssl.so.1 (0x00007f8a56220000)
libcrypto.so.1 => /lib64/libcrypto.so.1 (0x00007f8a55e47000)
libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f8a55c0e000)
libXinerama.so.1 => /usr/lib64/libXinerama.so.1 (0x00007f8a55a0b000)
libatk-1.0.so.0 => /usr/lib64/libatk-1.0.so.0 (0x00007f8a557e8000)
But -- it's all no good. Both chrome and chromium (which I can successfully compile from scratch without any problems), give the same blue screen of death with "Aw, Snap!" at start-up, and neither will even allow the display of the "about chrome page" or anything else. It's is 100% unable to display anything except popup dialogs. Please note: Firefox, several OpenGL games, and even GPU accerated ones like Trine, and all other normal slackware X11 applications work perfectly. There is no firewall, or ad-blocking, or anything installed on this system at the moment. Yet -- the problem persists.
I decided to enable the error reporting feature of chrome to let them know but that merely caused a list of error messages to scroll by in the terminal saying that the certificate of authority of google could not be verified locally... Which made me laugh, and decide to post here -- for Google probably believes there is nothing wrong with their browser under linux -- because they aren't receiveing any error messages... and never will...
So, how can I isolate what is causing the problem? (Or at very least get bug reporting working?!) I'm willing to try on chrome or chromium .
For my last attempt before giving up, on chromium, I tried assuming it was a gpu-related problem, and perhaps a kernel setting messed it up -- so I looked for clues on how to turn off the gpu completely.
I can not tell if these flags are specific to chrome, or if they will work with chrome and chromium both, but here's what I did from the command line:
chromium --disable-accelerated-composting --disable-accelerated-layers --disable-accelerated-2d-canvas --blacklist-webgl --blacklist-accelerated-composting --enable-logging=stderr --v=1 www.yahoo.com 2> ~/chrome_err.log
However, that didn't even get rid of the original error message that made me want to upgrade in the first place, for it still reports ERROR: multiple threads for gpu.
chrome_err.log:
[1725:1725:0324/120300:VERBOSE1:breakpad_linux.cc(1659)] Breakpad disabled
[1:1:0324/120300:VERBOSE1:zygote_main_linux.cc(600)] ZygoteMain: initializing 2 fork delegates
[1:1:0324/120300:VERBOSE1:nacl_fork_delegate_linux.cc(142)] NaClForkDelegate::Init()
[1:1:0324/120300:VERBOSE1:nacl_fork_delegate_linux.cc(142)] NaClForkDelegate::Init()
[1725:1748:0324/120300:VERBOSE1:google_api_keys.cc(237)] Using default value "726124521725-usiuc172onlf7t4ind7sf2detm950t7n.apps.googleusercontent.com" for API key GOOGLE_CLIENT_ID_MAIN
[1725:1748:0324/120300:VERBOSE1:google_api_keys.cc(237)] Using default value "ll1NK5GBOrAUb6zSbcgAX1Q7" for API key GOOGLE_CLIENT_SECRET_MAIN
[1725:1748:0324/120300:VERBOSE1:google_api_keys.cc(237)] Using default value "726124521725-usiuc172onlf7t4ind7sf2detm950t7n.apps.googleusercontent.com" for API key GOOGLE_CLIENT_ID_CLOUD_PRINT
[1725:1748:0324/120300:VERBOSE1:google_api_keys.cc(237)] Using default value "ll1NK5GBOrAUb6zSbcgAX1Q7" for API key GOOGLE_CLIENT_SECRET_CLOUD_PRINT
[1725:1748:0324/120300:VERBOSE1:google_api_keys.cc(237)] Using default value "726124521725-usiuc172onlf7t4ind7sf2detm950t7n.apps.googleusercontent.com" for API key GOOGLE_CLIENT_ID_REMOTING
[1725:1748:0324/120300:VERBOSE1:google_api_keys.cc(237)] Using default value "ll1NK5GBOrAUb6zSbcgAX1Q7" for API key GOOGLE_CLIENT_SECRET_REMOTING
[1725:1748:0324/120300:VERBOSE1:google_api_keys.cc(237)] Using default value "726124521725-usiuc172onlf7t4ind7sf2detm950t7n.apps.googleusercontent.com" for API key GOOGLE_CLIENT_ID_REMOTING_HOST
[1725:1748:0324/120300:VERBOSE1:google_api_keys.cc(237)] Using default value "ll1NK5GBOrAUb6zSbcgAX1Q7" for API key GOOGLE_CLIENT_SECRET_REMOTING_HOST
[1725:1725:0324/120300:VERBOSE1:pref_proxy_config_tracker_impl.cc(148)] 0x7fef8a158280: set chrome proxy config service to 0x7fef8a1899f0
[1725:1725:0324/120300:VERBOSE1:pref_proxy_config_tracker_impl.cc(277)] 0x7fef8a158280: Done pushing proxy to UpdateProxyConfig
[1725:1725:0324/120300:VERBOSE1:app_list_syncable_service_factory.cc(55)] AppListSyncableServiceFactory()
[1725:1748:0324/120300:VERBOSE1:multi_log_ct_verifier.cc(76)] Adding CT log: Google 'Pilot' log
[1725:1748:0324/120300:VERBOSE1:multi_log_ct_verifier.cc(76)] Adding CT log: Google 'Aviator' log
[1725:1748:0324/120300:VERBOSE1:multi_log_ct_verifier.cc(76)] Adding CT log: DigiCert Log Server
[1725:1725:0324/120300:VERBOSE1:bluetooth_low_energy_event_router.cc(192)] Initializing BluetoothLowEnergyEventRouter.
[1725:1725:0324/120300:VERBOSE1:bluetooth_low_energy_event_router.cc(195)] Bluetooth not supported on the current platform.
[1725:1725:0324/120300:VERBOSE1:app_list_syncable_service_factory.cc(45)] BuildInstanceFor: Default (0x7fef8a19bde0)
[1725:1725:0324/120300:VERBOSE1:pref_proxy_config_tracker_impl.cc(148)] 0x7fef8a287cb0: set chrome proxy config service to 0x7fef8a288370
[1725:1725:0324/120300:VERBOSE1:pref_proxy_config_tracker_impl.cc(277)] 0x7fef8a287cb0: Done pushing proxy to UpdateProxyConfig
[1725:1725:0324/120300:VERBOSE1:extension_service.cc(1533)] AddComponentExtension Bookmark Manager
[1725:1725:0324/120300:VERBOSE1:extension_service.cc(1533)] AddComponentExtension Cloud Print
[1725:1725:0324/120300:VERBOSE1:extension_service.cc(1533)] AddComponentExtension Web Store
[1725:1725:0324/120300:VERBOSE1:extension_service.cc(1533)] AddComponentExtension Chromium
[1725:1725:0324/120300:VERBOSE1:extension_service.cc(1533)] AddComponentExtension Settings
[1725:1725:0324/120300:VERBOSE1:extension_service.cc(1533)] AddComponentExtension Google Now
[1725:1725:0324/120300:VERBOSE1:extension_service.cc(1533)] AddComponentExtension CryptoTokenExtension
[1725:1725:0324/120300:VERBOSE1:app_list_syncable_service.cc(284)] 0x7fef8a26c8c0: AppListSyncableService: InitializeWithService.
[1725:1725:0324/120300:VERBOSE1:app_list_syncable_service.cc(837)] 0x7fef8a26c8c0: SyncStarted: Flare.
[1725:1725:0324/120300:WARNING:data_reduction_proxy_settings.cc(365)] SPDY proxy OFF at startup
[1725:1725:0324/120300:VERBOSE1:account_reconcilor.cc(70)] AccountReconcilor::AccountReconcilor
[1725:1725:0324/120300:VERBOSE1:account_reconcilor.cc(80)] AccountReconcilor::Initialize
[1725:1725:0324/120300:VERBOSE1:ev_whitelist_component_installer.cc(182)] Registering EV whitelist component.
[1725:1725:0324/120300:VERBOSE1:component_updater_service.cc(267)] CrxUpdateService starting up
[1725:1764:0324/120300:VERBOSE1:ev_whitelist_component_installer.cc(68)] Initial load: reading EV whitelist from file: /home/andrew3/.config/chromium/ev_hashes_whitelist.bin
[1725:1725:0324/120300:VERBOSE1:startup_browser_creator_impl.cc(587)] StartupBrowserCreatorImpl::ProcessStartupURLs
[1725:1751:0324/120300:VERBOSE1:ev_whitelist_component_installer.cc(147)] Verifying install: /home/andrew3/.config/chromium/EVWhitelist/6/_platform_specific/all/ev_hashes_whitelist.bin
[1725:1725:0324/120300:VERBOSE1:startup_browser_creator_impl.cc(595)] Pref: default
[1725:1764:0324/120300:VERBOSE1:packed_ct_ev_whitelist.cc(62)] Uncompressing EV whitelist of size 1113849
[1725:1751:0324/120300:VERBOSE1:ev_whitelist_component_installer.cc(159)] Whitelist size: 1113849
[1725:1744:0324/120300:VERBOSE1:crl_set_fetcher.cc(103)] Loaded 201209 bytes of CRL set from disk
[1725:1748:0324/120300:VERBOSE1:crl_set_fetcher.cc(125)] Installed CRL set #2146
[1725:1725:0324/120300:VERBOSE1:password_store_factory.cc(207)] Password storage detected desktop environment: (unknown)
[1725:1725:0324/120300:WARNING:password_store_factory.cc(237)] Using basic (unencrypted) store for password storage. See http://code.google.com/p/chromium/wiki/LinuxPasswordStorage for more information about password storage options.
[7:7:0324/120300:VERBOSE1:sandbox_linux.cc(64)] Activated seccomp-bpf sandbox for process type: renderer.
[7:7:0324/120300:VERBOSE1:child_thread.cc(245)] Mojo is disabled on child
[1725:1725:0324/120300:VERBOSE1:component_updater_service.cc(267)] CrxUpdateService starting up
[1725:1725:0324/120300:VERBOSE1:component_updater_service.cc(274)] First update attempt will take place in 360 seconds
[1725:1725:0324/120300:VERBOSE1:ev_whitelist_component_installer.cc(134)] Component ready, version 6 in /home/andrew3/.config/chromium/EVWhitelist/6
[1725:1742:0324/120300:VERBOSE1:ev_whitelist_component_installer.cc(36)] Reading new EV whitelist from file: /home/andrew3/.config/chromium/EVWhitelist/6/_platform_specific/all/ev_hashes_whitelist.bin
[1725:1764:0324/120300:VERBOSE1:ev_whitelist_component_installer.cc(88)] EV whitelist: Sucessfully loaded initial data.
[1725:1748:0324/120300:VERBOSE1:packed_ct_ev_whitelist.cc(26)] Setting new EV Certs whitelist.
[1725:1742:0324/120300:VERBOSE1:packed_ct_ev_whitelist.cc(62)] Uncompressing EV whitelist of size 1113849
[1725:1748:0324/120300:VERBOSE1:packed_ct_ev_whitelist.cc(26)] Setting new EV Certs whitelist.
[1754:1754:0324/120300:ERROR:sandbox_linux.cc(325)] InitializeSandbox() called with multiple threads in process gpu-process
[1754:1754:0324/120300:VERBOSE1:child_thread.cc(245)] Mojo is disabled on child
[12:12:0324/120300:VERBOSE1:sandbox_linux.cc(64)] Activated seccomp-bpf sandbox for process type: utility.
[12:12:0324/120300:VERBOSE1:child_thread.cc(245)] Mojo is disabled on child
[1725:1725:0324/120302:VERBOSE1:account_reconcilor.cc(98)] AccountReconcilor::Shutdown
[1725:1725:0324/120302:VERBOSE1:merge_session_helper.cc(212)] MergeSessionHelper::CancelAll
[1725:1725:0324/120302:VERBOSE1:account_reconcilor.cc(74)] AccountReconcilor::~AccountReconcilor
[1725:1725:0324/120302:VERBOSE1:ipc_sync_channel.cc(386)] Canceling pending sends
[1725:1725:0324/120302:VERBOSE1:component_updater_service.cc(286)] CrxUpdateService stopping
[1725:1727:0324/120302:VERBOSE1:sandbox_ipc_linux.cc(122)] SandboxIPCHandler stopping.
.... [Edit: Note -- I'm appending what happens when I manually type in a web page name, below]
[12:12:0324/121659:VERBOSE1:sandbox_linux.cc(64)] Activated seccomp-bpf sandbox for process type: utility.
[12:12:0324/121659:VERBOSE1:child_thread.cc(245)] Mojo is disabled on child
[14:14:0324/121702:VERBOSE1:sandbox_linux.cc(64)] Activated seccomp-bpf sandbox for process type: renderer.
[14:14:0324/121702:VERBOSE1:child_thread.cc(245)] Mojo is disabled on child
[1839:1862:0324/121702:VERBOSE1:resource_loader.cc(241)] OnReceivedRedirect: http://www.yahoo.com/
[1868:1915:0324/121702:VERBOSE1:ipc_sync_channel.cc(386)] Canceling pending sends
[1868:1868:0324/121702:VERBOSE1:ipc_sync_channel.cc(386)] Canceling pending sends
[1868:1915:0324/121702:VERBOSE1:ipc_sync_channel.cc(386)] Canceling pending sends
[1839:1839:0324/121705:VERBOSE1:account_reconcilor.cc(98)] AccountReconcilor::Shutdown
[1839:1839:0324/121705:VERBOSE1:merge_session_helper.cc(212)] MergeSessionHelper::CancelAll
[1839:1839:0324/121705:VERBOSE1:account_reconcilor.cc(74)] AccountReconcilor::~AccountReconcilor
[1839:1839:0324/121705:VERBOSE1:ipc_sync_channel.cc(386)] Canceling pending sends
[1839:1839:0324/121705:VERBOSE1:component_updater_service.cc(286)] CrxUpdateService stopping
****Edit: Note ... histograms would normally follow, but I truncated it.
So, anyone have an idea of how to figure out and repair the problem ?
I've never had bluetooth, so I don't think installing those libs will help.
But I am stumped and frustrated, there's no obvious error message clue as to what might be causing the problem, just some esoteric mention of "mojo" diabling without indication of whether that's even good or bad, and the fact that just about everything is shutting down and cancelling everything.... for no reason...
I solved the problem.
Running DMESG, I found a warning that an instruction exception had happened from Google Chrome. That's how I was able to figure out that the kernel processor configuration had somehow changed the memory management algorithms during my recompile.
I am using kernel 3.18.25, and I think it was specifically the "Processor Type and features -> Transparent Hugepage Suport" that caused the problem. Hugepage support must be set to "madvise" rather than to "always" in order for Chrome to work. I suspect the reason Chrome fails, is that the kernel call fails even though the memory is allocated when it is set to always. Some other people on the web reported a similar issue in obscure Google forums lists, so I'm just confirming that it is the most likely culprit.
Just in case that is not the only option which caused Google Chrome to fail, here are the kernel options which I tinkered with between the time I discovered the AW snap problem, and when the problem was solved.
I am listing only the options that I turned on, or changed the value of in that period of time. The rest of the unlisted options that deal with memory management, are all "off".
Processor Types and Features -> Linux guest support: (enable)
Enable paravirtualization, Paravirtualization layer for spinlocks, Xen guest support, Support for running as PVH guest, KVM Guest support, Paravirtualization steal time accounting.
Processor Types and Features -> Support Intel Processors, Support AMD processor
Processor Types and Features:
Enable 1GB pages for kernel pagetables.
Numa Memory allocation and Scheduler support:
ACPI NUMA detection (only),
6 (Maximum Numa nodes as a power of 2),
Sparse Memory virtual memmap,
Enable to assign a node which has only movable memory,
enable for balloon memory compaction,
page migration,
Enable KSM for page merging,
(4096) Low address space protect from user allocation,
Transparent Hugepage support (madvise),
Continuguous memory allocator,
(7) maximum count of the CMA areas,
MTRR support, cleanup support,
MTRR cleanup enable value (1),
MTRR cleanup spare reg num (1),
Enable seccompt to safely compute untrusted byte code.
Try to start it as chromium --no-sandbox - worked for me (but then I guess my kernel somewhat misconfigured)

Why doesn't try/catch work in Matlab when the Image Acquisition Toolbox drops frames?

I am using Matlab's Image Acquisition Toolbox to acquire high-speed video over gigabit Ethernet. I'm having some trouble with frame-dropping, but that's not what this question is about. What I really want to do is tell Matlab to continue running the script even after encountering the frame-dropping error.
I used a try/catch statement for this purpose but it just doesn't work. Here is my code, sparing some of the details relating to setting up the camera and using the data:
%% setting up camera
while(1)
% continue acquiring data forever
while(vidObj.FramesAvailable < vidObj.FramesPerTrigger)
% wait until we're ready to get the data
try
pause(.1)
catch exception
disp "i got an error"
end
end
% get the data
[img, t] = getdata(vidObj);
%% do something with the data
%% ...
end
What happens is that, every once in a while, some frames are dropped and the toolbox raises an error. This happens inside the try block, but Matlab raises an exception anyway! The output looks something like:
Error event occurred at 21:08:20 for video input object: Mono8-gige-1.
gige: Block/frame 1231 is being dropped beecause a lost packet is unable to be resent....
Error in script_name (line 82)
pause(.1)
You can see that the error occurs while we're waiting to collect data (the "pause" statement), which is inside the try block, and yet the exception is not caught correctly because my debugging message doesn't print and the program grinds to a halt.
How can I get Matlab to observe the try/catch structure and continue after this error happens?
I figured it out. The error message is not a true error, but more of a warning. Execution does not stop. However, vidObj stops collecting frames and my code keeps looping forever, waiting for enough frames to be collected.
You can insert a check for this condition like so:
% wait until enough frames are available
while(vidObj.FramesAvailable < vidObj.FramesPerTrigger)
pause(.1)
if strcmp(vidObj.Running, 'off')
% It has stopped running, probably because frames were dropped
start(vidObj)
end
end
Now, upon frame dropping, the object will be restarted and acquistion continues. Obviously the dropped frames cannot be recovered so there will be a gap in the video.