I'm using Chrome's performance tab to study the performance of a page, and I occasionally get a warning like:
DevTools: CPU profile parser is fixing 4 missing samples.
Does anyone know what this means? Googling for this warning has returned no results so far...
As coming across with and having this situation, possible helpful things to consider are as below.
As Chrome 58 is released in 2017, some changes are done related to the analyzing performance. For example:
Timeline panel is renamed as Performance panel.
Profiles panel is renamed as Memory panel.
Record Javascript CPU Profile menu is moved into Dev Tools → Three dots at right → More tools → Javascript Profiler. (Old version of Javascript Profiler)
In addition of these, the warning message that is seen (DevTools: CPU profile parser is fixing N missing samples.) is being written to the console window when there is single (program) sample between two call stacks sharing the same bottom node. Also, the samples count should be greater than or equal to 3 according to the source code.
Comments written above _fixMissingSamples method in the CPUProfileDataModel.js file explains this situtation as below;
// Sometimes sampler is not able to parse the JS stack and returns
// a (program) sample instead. The issue leads to call frames belong
// to the same function invocation being split apart.
// Here's a workaround for that. When there's a single (program) sample
// between two call stacks sharing the same bottom node, it is replaced
// with the preceeding sample.
In the light of these information, we can trace the code and examine the behavior.
CPUProfileDataModel.js
let prevNodeId = samples[0];
let nodeId = samples[1];
let count = 0;
for (let sampleIndex = 1; sampleIndex < samplesCount - 1; sampleIndex++) {
const nextNodeId = samples[sampleIndex + 1];
if (nodeId === programNodeId && !isSystemNode(prevNodeId) && !isSystemNode(nextNodeId) &&
bottomNode(idToNode.get(prevNodeId)) === bottomNode(idToNode.get(nextNodeId)) {
++count;
samples[sampleIndex] = prevNodeId;
}
prevNodeId = nodeId;
nodeId = nextNodeId;
}
if (count) {
Common.console.warn(ls`DevTools: CPU profile parser is fixing ${count} missing samples.`);
}
It seems that, it simply compares previous and next node related to current node as if they has the same bottom node (actually comparing parent nodes). Also previous and next node shouldn't be a system (program/gc/idle function) node and current node should be the 'program' node. If it is the case, then the current node in samples array is set to previous node.
idle: Waiting to do process
program: Native code execution
garbage collector: Accounts for Garbage Collection
Also, disabling Javascript samples from Performance → Capture Settings result fewer details & call stacks because of omitting all the call stacks. The warning message shouldn't appear in this case.
But, since this warning is about the sampler that says cannot parse the JS stack and call frames being split apart, it doesn't seem very important thing to consider.
Resources:
https://github.com/ChromeDevTools/devtools-frontend/tree/master/front_end/sdk
https://github.com/ChromeDevTools/devtools-frontend/blob/master/front_end/sdk/CPUProfileDataModel.js
https://chromium.googlesource.com/chromium/blink/+/master/Source/devtools/front_end/sdk/
https://developers.google.com/web/tools/chrome-devtools/evaluate-performance
https://developers.google.com/web/updates/2016/12/devtools-javascript-cpu-profile-migration
Related
Say I have inside my JRuby program the following loop:
loop do
x=foo()
break if x
bar()
end
and I want to collect profiling information just for the invocations of bar. How to do this? I got so far:
pd = []
loop do
x=foo()
break if x
pd << JRuby::Profiler.profile { bar() }
end
This leaves me with an array pd of profile data objects, one for each invocation of bar. Is there a way to create a "summary" data object, by combining all the pd elements? Or even better, have a single object, where profile would just add to the existing profiling information?
I googled for a documentation of the JRuby::Profiler API, but couldn't find anything except a few simple examples, none of them covering my case.
UPDATE : Here is another attempt I tried, which does not work either.
Since the profile method initially clears the profile data inside the Profiler, I tried to separate the profiling steps from the data initializing steps, like this:
JRuby::Profiler.clear
loop do
x=foo()
break if x
JRuby::Profiler.send(:current_thread_context).start_profiling
bar()
JRuby::Profiler.send(:current_thread_context).stop_profiling
end
profile_data = JRuby::Profiler.send(:profile_data)
This seems to work at first, but after investigation, I found that profile_data then contains the profiling information from the last (most recent) execution of bar, not of all executions collected together.
I figured out a solution, though I have the feeling that I'm using a ton of undocumented features to get it working. I also must add that I am using (1.7.27), so later JRuby versions might or might not need a different approach.
The problem with profiling is that start_profiling (corresponding to the Java method startProfiling in the class Java::OrgJrubyRuntime::ThreadContext) not only turns on the profiling flag, but also allocates a fresh ProfileData object. What we want to do, is to reuse the old object. stop_profiling OTOH only toggles the profiling switch and is uncritical.
Unfortunately, ThreadContext does not provide a method to manipulate the isProfiling toggle, so as a first step, we have to add one:
class Java::OrgJrubyRuntime::ThreadContext
field_writer :isProfiling
end
With this, we can set/reset the internal isProfiling switch. Now my loop becomes:
context = JRuby::Profiler.send(:current_thread_context)
JRuby::Profiler.clear
profile_data_is_allocated = nil
loop do
x=foo()
break if x
# The first time, we allocate the profile data
profile_data_is_allocated ||= context.start_profiling
context.isProfiling = true
bar()
context.isProfiling = false
end
profile_data = JRuby::Profiler.send(:profile_data)
In this solution, I tried to keep as close as possible to the capabilities of the JRuby::Profiler class, but we see, that the only public method still used is the clear method. Basically, I have reimplemented profiling in terms of the ThreadContext class; so if someone comes up with a better way to solve it, I will highly appreciate it.
The Reference Source page for stringbuilder.cs has this comment in the ToString method:
if (chunk.m_ChunkLength > 0)
{
// Copy these into local variables so that they
// are stable even in the presence of ----s (hackers might do this)
char[] sourceArray = chunk.m_ChunkChars;
int chunkOffset = chunk.m_ChunkOffset;
int chunkLength = chunk.m_ChunkLength;
What does this mean? Is ----s something a malicious user might insert into a string to be formatted?
The source code for the published Reference Source is pushed through a filter that removes objectionable content from the source. Verboten words are one, Microsoft programmers use profanity in their comments. So are the names of devs, Microsoft wants to hide their identity. Such a word or name is substituted by dashes.
In this case you can tell what used to be there from the CoreCLR, the open-sourced version of the .NET Framework. It is a verboten word:
// Copy these into local variables so that they are stable even in the presence of race conditions
Which was hand-edited from the original that you looked at before being submitted to Github, Microsoft also doesn't want to accuse their customers of being hackers, it originally said races, thus turning into ----s :)
In the CoreCLR repository you have a fuller quote:
Copy these into local variables so that they are stable even in the presence of race conditions
Github
Basically: it's a threading consideration.
In addition to the great answer by #Jeroen, this is more than just a threading consideration. It's to prevent someone from intentionally creating a race condition and causing a buffer overflow in that manner. Later in the code, the length of that local variable is checked. If the code were to check the length of the accessible variable instead, it could have changed on a different thread between the time length was checked and wstrcpy was called:
// Check that we will not overrun our boundaries.
if ((uint)(chunkLength + chunkOffset) <= ret.Length && (uint)chunkLength <= (uint)sourceArray.Length)
{
///
/// imagine that another thread has changed the chunk.m_ChunkChars array here!
/// we're now in big trouble, our attempt to prevent a buffer overflow has been thawrted!
/// oh wait, we're ok, because we're using a local variable that the other thread can't access anyway.
fixed (char* sourcePtr = sourceArray)
string.wstrcpy(destinationPtr + chunkOffset, sourcePtr, chunkLength);
}
else
{
throw new ArgumentOutOfRangeException("chunkLength", Environment.GetResourceString("ArgumentOutOfRange_Index"));
}
}
chunk = chunk.m_ChunkPrevious;
} while (chunk != null);
Really interesting question though.
Don't think that this is the case - the code in question copies to local variables to prevent bad things happening if the string builder instance is mutated on another thread.
I think the ---- may relate to a four letter swear word...
I read wikipedia page about heisunbug, but don't understand this example. Can anyone explain it in detail?
Debuggers also commonly provide watches or other user interfaces that cause additional source code (such as property accessors) to be executed stealthily, which can, in turn, change the state of the program.
I think what it's getting at is that the debugger itself may call code (such as getters) to retrieve the value of a property you might have placed a watch on.
Consider the getter:
def getter fahrenheit:
return celsius * 9 / 5 + 32;
and what would happen if you put a watch on the fahrenheit property.
That code would normally only be called if your code itself tried to access the fahrenheit propery but, if a debugger is calling it to maintain the watch, it may be called outside of the control of your program.
A simple example, let's say the getter has a (pretty obvious) bug which means that it returns the wrong result the first time it's called:
class temperature:
variable state
def init:
state = 1
def getter fahrenheit:
if state == 1:
state = 0
return -42
return celsius * 9 / 5 + 32;
So running your code without a debugger exhibits a problem in that it will return a weird value the first time your code calls it.
But, if your debugger is actually calling the getter to extract a value that it's watching (and it's probably doing this after every single-step operation you perform), that means the getter will be well and truly returning the correct value by the time your code calls it for what it thinks is the first time.
Hence the problem will disappear when you try to look closer at it, and that's the very definition of a Heisenbug, despite the fact that Heisenberg's actual uncertainty principle has little to do with the observer effect.
I'm using Chrome (Version 19.0.1084.46). I enabled the Gamepad API in chrome://flags. I plugged in some game pads, but navigator.webkitGamepads is always an array of length 4 containing only undefined.
navigator.webkitGamepads
GamepadList
0: undefined
1: undefined
2: undefined
3: undefined
length: 4
__proto__: GamepadList
What do I need to do to test out using gamepads? I'm on Ubuntu Linux if that matters.
I was having trouble with this as well (on Ubuntu 10.04 with Chrome 21.0.1163.0 dev). I ran across this from a thread on chromium-discussions:
Note that you need to press a face button on the gamepad before data
will be available. This is due to fingerprinting concerns.
I wrote a quick test page that seems to work if you hold a controller button down while refreshing the page. I'm using a Gamestop-branded Xbox 360 wired controller with the xboxdrv driver.
Also, one other important thing to note - Chrome treats these Gamepad objects like snapshots of the controller state. So if you pass around the actual Gamepad object, you'll never get updated information. Chrome seems to only poll the controller when you call the navigator.webkitGamepads[x] getter (see line 23 in my test page).
SPECIAL NOTE: The GamePad API is handled in two completely different ways by Firefox and Google Chrome. Because of this, you need to include code to test for your browser's identity, and handle the cases accordingly.
When the navigator.getGamePads() method is invoked, an array of objects is returned, and those objects contain information about the identity of your gamepads/joysticks, and the state of the axes/buttons thereof.
Here's the issue: on Firefox, the method hands the array to your calling code, and you can then examine that array repeatedly, to look for changes of axis/button state in your own code. Firefox behaves in the expected manner, and updates the objects you've received with new axis/button data, as and when it becomes available.
Google Chrome, on the other hand, treats the array totally differently. Chrome treats the array as a one-time snapshot of the status of your gamepads/joysticks, and if you want to fetch new data, you have to perform another invocation of navigator.getGamePads(), to fetch a new snapshot, discarding the old one.
As a consequence, you need something like this to make your code work on both browsers (assuming that the variable 'gp' is the variable in which you previously stored the return value from your first invocation of navigator.getGamePads() ...)
var ua = navigator.userAgent;
if (ua.toLowerCase().indexOf("chrome") != -1)
gp = navigator.getGamePads();
Why the two browsers behave so differently, is a topic for another thread, but I suspect it has much to do with the fact that the GamePad API is still somewhat experimental, and the fine detail of the standard applicable thereto has not yet been finalised.
In my case, running Chrome on Windows 7 64-bit (but the 32-bit version, for some strange reason best known to my computer's manufacturer, who installed it in this fashion), the array returned by navigator.getGamePads() currently contains only as many entries as there are actual gamepads/joysticks plugged into my computer. If you're experiencing something different on Chrome under Linux, then you need to take this into account also, by testing to see if you have any null values in the array.
Another factor to take into account, is that when selecting a gamepad/joystick, the standard in place calls for the index property of the GamePad object to be referenced, and used thereafter to index into the array. The reason for this is covered in more detail in this document:
W3C page : Editor's Draft on the GamePad interface
However, whilst the implementation of this works as documented above in my incarnation of Google Chrome, you may have to check your version experimentally to see if it behaves the same as mine. If it doesn't, adjust your code accordingly until such time as the standard is finalised, and proper conformity thereto is enforced in all modern browsers and all OS incarnations thereof.
So, when you're selecting a particular gamepad/joystick to work with, the code will look something like this (with the variable 'gid' as your index selector):
var gLen = this.gamePads.length;
done = false;
idx = 0;
while (!done)
{
if (this.gamePads[idx] !== null)
{
if (gid == this.gamePads[idx].index)
{
//Here, choose this GamePad as the selected GamePad, based upon the index number as per the W3C recommendation ...
selectedGamePadIndex = gid;
selectedGamePad = this.gamePads[idx];
done = true;
//End if
}
//End if
}
//Update counter in case we haven't selected one of the available GamePads above ...
idx++;
if (idx >= gLen)
done = true; //Exit loop altogether if we've exhausted the list of available GamePads
//End while
}
I actually implemented the above code (along with some tidying up at the end) as a method for a custom object, but the principle remains the same regardless of the implementation minutiae. (I'll confess at this juncture that I have a habit of writing my own short libraries to handle tasks like this!)
Another issue to watch out for, is the manner in which the gamepad API maps the physical data sources of your gamepad/joystick to the axis/button elements of the GamePad object, and the values contained therein. I have a Microsoft Sidewinder Pro USB joystick, and the mappings for this device on my computer are seriously weird, viz:
Axes 0/1: Joystick handle (as expected)
Axes 2/3: Not assigned
Axes 4/5: 4 not assigned, 5 assigned to twist grip
Axes 6/7: 6 assigned to round slider, 7 not assigned
Axes 8/9: 8 not assigned, 9 assigned to hat switch (er, WHAT???)
Yes, that's right, analogue axis 9 is assigned to the hat switch, and seriously odd values are returned, viz:
Hat switch centred: +1.2
Hat switch up: -1.0
Hat switch down: +0.1
Hat switch left: +0.7
Hat switch right: -0.42
Quirks like this are something you should be alert to as you experiment with the GamePad API. Even worse, a so-called "standard" GamePad (defined in that document I linked to above) may report as standard on some browsers, but not on others, or worse still, report as standard on browser X in Windows, but not on the same browser X in Linux! While this will induce much hair-tearing frustration as you try to fight your way through this software equivalent of bramble thicket, getting badly scratched by some of the thorns along the way, you can at least take comfort in the fact that browser developers are working to try and ameliorate this, but it'll take time, because other things take priority (such as making sure your browser doesn't become an easy target for ransomware to hijack, which will ruin you day immensely if it happens).
This didn't work for me but maybe it helps you? Pulled from here.
function updateStatus() {
window.webkitRequestAnimationFrame(updateStatus);
var gamepads = navigator.webkitGamepads;
var data = '';
for (var padindex = 0; padindex < gamepads.length; ++padindex)
{
var pad = gamepads[padindex];
if (!pad) continue;
data += '<pre>' + pad.index + ": " + pad.id + "<br/>";
for (var i = 0; i < pad.buttons.length; ++i)
data += "button" + i + ": " + pad.buttons[i] + "<br/>";
for (var i = 0; i < pad.axes.length; ++i)
data += "axis" + i + ": " + pad.axes[i] + "<br/>";
}
document.body.innerHTML = data;
}
window.webkitRequestAnimationFrame(updateStatus);
Or as a hard alternative, there's the Javascript Joystick Plug-in (demo here) but I don't think that works in Linux.
I'm doing a little research on possible application of EWS in our existing project which is written with heavy use of MAPI and I found out something disturbing about performance of LoadPropertiesForItems() method.
Consider such scenario:
we have 10000 (ten thousands) messages in Inbox folder
we want to get approximately 30 properties of every message to see if they satisfy our conditions for further processing
messages are retrieved from server in packs of 100 messages
So, code looks like this:
ItemView itemsView = new ItemView(100);
PropertySet properties = new PropertySet();
properties.Add(EmailMessageSchema.From);
/*
add all necessary properties...
*/
properties.Add(EmailMessageSchema.Sensitivity);
FindItemsResults<Item> findResults;
List<EmailMessage> list = new List<EmailMessage>();
do
{
findResults = folder.FindItems(itemsView);
_service.LoadPropertiesForItems(findResults, properties);
foreach (Item it in findResults)
{
... do something with every items
}
if (findResults.NextPageOffset.HasValue)
{
itemsView.Offset = findResults.NextPageOffset.Value;
}
}while(findResults.MoreAvailable);
And the problem is that every increment of itemsView.Offset property makes LoadPropertiesForItems method longer to execute. For first couple of iterations it is not very noticeable but around 30th time loop makes that call time increases from under 1 second to 8 or more seconds. And memory allocation hits physical limits causing out of memory exception.
I'm pretty sure that my problems are "offset related" because I changed a code a little to that:
itemsView = new ItemView(100, offset, OffsetBasePoint.Beginning);
...rest of loop
if (findResults.NextPageOffset.HasValue)
{
offset = findResults.NextPageOffset.Value;
}
and I manipulated offset variable (declared outside of loop) in that way that I set its value on 4500 at start and than in debug mode after first iteration I changed its value to 100. And according to my suspicions first call of LoadPropertiesForItems took veeeery long to execute and second call (with offset = 100) was very quick.
Can anyone confirm that and maybe propose some solution for that?
Of course I can do my work without using an offset but why should I? :)
Changing the offset is expensive because the server has to iterate through the items from the beginning -- it isn't possible to have an ordinal index for messages because new messages can be inserted into the view in any order (think of a view over name or subject).
Paging through all the items once is the best approach.