Warning: as the reference types proposal isn't complete yet, this code will not run without toggling flags or setting in order to enable executing experimental code.
If you are on Google Chrome or a Chromium browser, you will need to enable the following flag:
chrome://flags/#enable-experimental-webassembly-features
I had set up a simple handwritten Wasm module for personal use. I could've easily written it in JavaScript, but it was easier and made more sense to use Wasm, and since it was a simple, personal script, I wouldn't care if other people couldn't run it.
I had compiled it using wabt's wat2wasm.
The Wasm module was intended to be fed the entire globalThis object to import from.
From there, it took four TypedArray constructors: Uint8Array, Uint16Array, Uint32Array, and BigUint64Array.
Take note: no code was executed prior to the Wasm, thus there cannot be any interference.
Later, I had realized that that the Wasm wasn't working as intended at all, my math was correct, but the variables were wrong.
I had narrowed my problem down to just this:
;; global -> global variable
(import "globalThis" "Uint8Array" (global $Uint8Array externref))
(import "globalThis" "Uint16Array" (global $Uint16Array externref))
(import "globalThis" "Uint32Array" (global $Uint32Array externref))
(import "globalThis" "BigUint64Array" (global $BigUint64Array externref))
;; func -> function
(import "console" "log" (func $console::log (param externref)))
(start $_start)
(func $_start
global.get $Uint8Array
call $console::log
global.get $Uint16Array
call $console::log
global.get $Uint32Array
call $console::log
global.get $BigUint64Array
call $console::log
)
This Wasm was instantiated like so:
WebAssembly.instantiateStreaming(
fetch(
"test.wasm", {
mode : "same-origin",
credentials : "omit",
cache : "no-store",
redirect : "error",
referrer : "no-referrer"
}
), globalThis
).catch( console.error );
The interesting problem is that the logs all say the same thing: Uint8Array.
I was dumbfounded. This has to be literally impossible. The JavaScript file itself was not cached, the WebAssembly was being fetched with "no-cache," the web page itself wasn't cached.
Then I thought, because I was using XHTML, maybe it didn't happen in HTML files. It did there too.
I tried converting the Wasm file to a TypedArray and just using WebAssembly.instantiate, suddenly, it worked flawlessly.
At this point the server must be serving the wrong file, because the further I go, the more this seems like nonsense.
I almost want to say that this is a Chromium browser or V8 runtime error, but I need to narrow it down a bit more before I blindly attempt to present this as a bug.
I have set up two different versions of the same code, on Repl.it, and on CodeSandBox.io, so that hopefully someone can try running it themself, to confirm the bug, and maybe attempt to debug where I went wrong.
(Could this be a problem with Repl.it's server?)
This was a compiler bug fixed by Chromium in
https://chromium-review.googlesource.com/c/v8/v8/+/2551100.
This was the response that I had gotten from one of the developers:
This is indeed a timing issue that has been fixed in https://chromium-review.googlesource.com/c/v8/v8/+/2551100. The problem happens when there are only imported globals, and compilation of the WebAssembly functions finishes before the stream actually finishes. In this case, the offset calculation happens after the compiler uses the offsets, and therefore produces incorrect code.
A workaround is to define one global that is not imported, as this causes the offset calculation to happen earlier.
Seems like sending a small module that only imports globals instead of functions was breaking the code.
Their code had a threading race condition between the stream and the compiler.
Related
When working on a SolidJS project you might start seeing the following warning message in your JS console:
computations created outside a `createRoot` or `render` will never be disposed
There are some information available on this in SolidJS' Github repository issues. But after reading them I was still not quite sure what this was all about and whether my code was really doing something wrong.
I managed to track down where it came from and find a fix for it based on the documentation. So I'm providing the explanation and the solution for those Googling this warning message.
In essence this is a warning about a possibility of a memory leak due to a reactive computation being created without the proper context which would dispose of it when no longer needed.
A proper context is created a couple of different ways. Here are the ones I know about:
By using the render function.
By using the createRoot function. Under the hood render uses this.
By using the createContext function.
The first is by far the most common way, because each app has at least one render function call to get the whole show started.
So what makes the code go "out of context"?
Probably the most common way is via async calls. The context creation with its dependency tree happens only when the synchronous portion of the code finishes running. This includes all the export default function in your modules and the main app function.
But code that runs at a later time because of a setTimeout or by being in an async function will be outside of this context and any reactive computations created will not be tracked and might stick around without being garbage collected.
An example
Let's say you have a data input screen and have a Save button on it that makes an API call to your server to save the data. And you want to provide a feedback to the user whether the operation succeeded or not, with a nice HTML formatted message.
[msg,setMsg] = createSignal(<></>)
async function saveForm(){
...
setMsg(<p>Saving your data.<i>Please stand by...</i></p>)
const result=await callApi('updateUser',formData)
if(result.ok){
setMsg(<p>Your changes were <b>successfully</b> saved!</p> )
} else {
setMsg(<p>There was a problem saving your data! <br>Error: </p><pre>{result.error}</pre> )
}
}
...
<div>
...
<button onClick={saveForm} >Save</button>
{msg()}
</div>
This will produce the above mentioned warning when the API call returns an error, but not the other times. Why?
The reason for this is that SolidJS considers the code inserts inside JSX to be reactive, ie: need to be watched and re-evaluated. So inserting the error message from the API call creates a reactive computation.
The solution
I found the solution at the very end of the SolidJS doc. It's a special JSX modifier: /*#once*/
It can be used at the beginning of a curly brace expression and it tells the SolidJS compiler to explicitly not to make this a reactive expression. In other words: it will evaluated once and only once when the DOM nodes are created from the JSX.
In the above example here's how to use it:
setMsg(<p>There was a problem saving your data! <br>Error: </p><pre>{/*#once*/ result.error}</pre> )
After this there will be no more warning messages :)
In my case, I had an input and when that input changed I re-created an SVG drawing. Because the SVG creation was an expensive operation, I added a debounce in the createEffect function which ran when the input changed. debounce is a technique to defer the processing until the input stops changing for at least X amount of time. It involved running the SVG generation code inside the setTimeout function, thus being outside of the main context. Using the /*#once*/ modifier everywhere where I inserted an expression in the generated JSX has fixed the problem.
when I reverse the binary with IDA gui, all the functions get decompiled without a problem.
but when I am running an automatic script on ida without gui, there is always the same function, that refuses to be decompiled. (when I am openning the same IDB that the automation script worked on, the function get decompiled without a problem)
I am using bip. and using BipFunc.can_decompile to check if a function can get decompiled.
EDIT:
according to an answer bellow, I have tried to add the following:
if not func.can_decompile:
print(f"can't decompile function 0x{func.ea:04x}, trying again")
decomp_all()
if not func.can_decompile:
print(f"can't decompile function 0x{func.ea:04x}, trying again")
decomp_all_twice_cacheclear()
if not func.can_decompile:
print(f"can't decompile function 0x{func.ea:04x}, skipping...")
return
sadly it did not work, I get all 3 prints every time, even on different binaries
it seems to be fixed on IDA Pro 7.6
There is several reason you can get an error on the decompilation from IDA. If it works on some case and other it does not it is probably because of the call analysis. When decompiling a function IDA will try to gather information on the function called by this one and in some case fail to get those information which will make the decompilation fail. But once that function has been decompiled, the information fetched by IDA will be updated, and so the decompilation of the caller function might now work. So basically it means you have to decompile the function in an order, which is a pain, for fixing that the simplest way is to just decompile everything twice, but it can take quite some time if you do it on "big" binaries.
I though I put that in the Bip repository somewhere but I can't find it, so here is a small plugin/code which should allows to do that:
from bip import *
class DecompileAll(BipPlugin):
"""
Plugin for decompiling all the function in the binary.
"""
#menu("Bip/DecompileAll/", "Invalidate hexrays caches")
def clear_hxcCache(self):
HxCFunc.invalidate_all_caches()
#menu("Bip/DecompileAll/", "Decompile all func")
def decomp_all(self):
count = 0
for f in HxCFunc.iter_all():
count += 1
print("0x{:X} functions decompiled".format(count))
#menu("Bip/DecompileAll/", "Decompile twice with cache clear")
def decomp_all_twice_cacheclear(self):
HxCFunc.invalidate_all_caches()
self.decomp_all()
self.decomp_all()
Just for information the basic reason for decompilation error, is that it is not able to make a correct translation of some piece of code because it does not understand the assembly, this is typically true if there is a problem during the analysis and the code is not correctly detected (also happens a lot if you are dealing with obfuscation). You can typically view this case by an error telling you "failed analysis at ADDR" in the IDAPython console, and then look at the problem. Probably not your case but might still help.
Glad to hear you are using bip. So about the BipFunc.can_decompile function: like indicated in the documentation (https://synacktiv.github.io/bip/build/html/base/func.html#bip.base.BipFunction.can_decompile) it will just try to decompile the function and see if an error occurs. The code is pretty straight forward (https://github.com/synacktiv/bip/blob/master/bip/base/func.py#L372), this is mostly written for being done while using one-liner, its the same thing as catching the exception when trying to decompile.
I'm trying to deploy an app to production and getting a little confused by environment and application variables and what is happening at compile time vs runtime.
In my app, I have a genserver process that requires a token to operate. So I use config/releases.exs to set the token variable at runtime:
# config/releases.exs
import Config
config :my_app, :my_token, System.fetch_env!("MY_TOKEN")
Then I have a bit of code that looks a bit like this:
defmodule MyApp.SomeService do
use SomeBehaviour, token: Application.get_env(:my_app, :my_token),
other_config: :stuff
...
end
In production the genserver process (which does some http stuff) gives me 403 errors suggesting the token isn't there. So can I clarify, is the use keyword getting evaluated at compile time (in which case the application environment doest exist yet)?
If so, what is the correct way of getting runtime environment variables in to a service like this. Is it more correct to define the config in application.ex when starting the process? eg
children = [
{MyApp.SomeService, [
token: Application.get_env(:my_app, :my_token),
other_config: :stuff
]}
...
]
Supervisor.start_link(children, opts)
I may have answered my own questions here, but would be helpful to get someone who knows what they're doing confirm and point me in the right way. Thanks
elixir has two stages: compilation and runtime, both written in Elixir itself. To clearly understand what happens when one should figure out, that everything is macro and Elixir, during compilation stage, expands these macros until everything is expanded. That AST comes to runtime.
In your example, use SomeBehaviour, foo: :bar is implicitly calling SomeBehaviour.__using__/1 macro. To expand the AST, it requires the argument (keyword list) to be expanded as well. Hence, Application.get_env(:my_app, :my_token) call happens in compile time.
There are many possibilities to move it to runtime. If you are the owner of SomeBehaviour, make it accept the pair {:my_app, :my_token} and call Application.get_env/2 somewhere from inside it.
Or, as you suggested, pass it as a parameter to children; this code belongs to function body, meaning it won’t be attempted to expand during compilation stage, but would rather be passed as AST to the resulting BEAM to be executed in runtime.
I have C+ program that compiles to web assebmbly using the emscripten system. I would like to clean up some things, flush files, etc etc. when he page running the program is closed.
in main there is:
emscripten_set_main_loop_arg(onMainLoopTick, arg, 0, 1);
Currently when the page closes the "process" is simply exited and does not continue after the "loop simulator". I figure I need to get an event from the page that will block the main thread until the C++ code process it and cleans up it's mess.
What event should I forward to C++ and how should I use it ?
The first things to know is that there is no native library nor APIs for WebAssembly (I mean..yet, as of MVP. There are native features like threads coming as post-MVP feature). What is means that all system libraries in C++ are implemented by importing emulated JavaScript functions. So if you are looking for native features like detecting closing events, you should check if there is JS/HTML5 APIs that do the similar things.
To see how it works, open generated .wast file and search for import instructions and generated JS files. Also, you may want to search on Emscripten repo directly to check if there is JS/HTML5 bindings available on C++ side, as their documentation is quite large and hard to look through.
Sticking to the point, the HTML5 events that are fired when closing are beforeunload and unload. I would prefer using beforeunload event. Emscripten provides em_beforeunload_callback callback function type and emscripten_set_beforeunload_callback to register in html5.h bindings.
Otherwise, you use them directly. For example:
In C++:
void EMSCRIPTEN_KEEPALIVE clean_stuff() {
// Clean up the mess...
// You should use EMSCRIPTEN_KEEPALIVE or
// add it to EXPORTED_FUNCTIONS in emcc compilation options
// to make it callable in JS side.
}
In JS:
window.addEventListener("beforeunload", function (event) {
// Exported functions are prefixed by an underscore
Module._clean_stuff();
});
I load some textures in asynchronously at the beginning of my game, about 40-50 of them.
vector<string> textureFileNames;
textureFileNames.push_back("textures/particle.png");
textureFileNames.push_back("textures/menu_title.png");
...
textureFileNames.push_back("textures/timer_bar.png");
for (auto fileName: textureFileNames)
{
Director::getInstance()->getTextureCache()
->addImageAsync(fileName, CC_CALLBACK_1(LoadingLayer::textureLoadedCallback, this));
}
My textureLoadedCallback method does nothing funky; at this stage it simply increments a value and updates a progress timer. The callback is called from the main thread by cocos2d-x design, so I don't suspect any problems arise from there.
90% of the time this works fine. But sometimes it crashes in VS2012 midway through loading the textures:
Debug Assertion Failed!
Program: C:\Windows\system32\MSVCP110D.dll
File: C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\include\vector
Line: 1140
Expression: vector subscript out of range
Breaking at this point, I can see that it dies in the internals of vector, specifically the [] operator, and traces back through _Hash to the TextureCache::loadImage() method: auto it = _textures.find(asyncStruct->filename) on line 174 of CCTextureCache.cpp. _textures is defined as std::unordered_map<std::string, Texture2D*> _textures, a standard library unordered map. asyncStruct->filename resolves to the full path and filename of a texture to load.
Using the debugger, I can see that the filename is fine. I can see that _textures already contains the 19 textures before this one that it has processed just fine.
The fact that it seems to just be dying in the midst of the standard library doesn't strike me as right... but I'm unable to determine where CCTextureCache goes wrong. Only that it doesn't always fail, and that it's failing in an asynchronous thread. There's no concurrency bollocks going on with my code (as far as I know).
Is this a cocos2d-x bug, a VS2012 bug or a bug with my code I pasted above?
I think a potential cause could be that the for loop issues like 19 async image loads all at once, which may or may not be supported by that method. Try issuing the next async load only after your texture callback is called. That way no two async loads are performed simultaneously.