I don't want to use --no-sandbox for security reasons, but without it, I can't use --no-zygote which is the only solution I could find to prevent zombie process from being created. How can I achieve the same goal of cleaning up zombie processes without --no-sandbox? I know about dumb-init, but I want to know if there is a way to keep the processet from becoming zombies in the first place.
The zombie processes left behind are like this
$ ps aux | grep chrome | head -n 10
app 60 0.0 0.0 0 0 ? Z Mar24 0:00 [chrome_crashpad] <defunct>
app 65 0.0 0.0 0 0 ? Z Mar24 0:00 [chrome] <defunct>
app 66 0.0 0.0 0 0 ? Z Mar24 0:00 [chrome] <defunct>
app 82 0.0 0.0 0 0 ? Z Mar24 0:00 [chrome] <defunct>
app 163 0.0 0.0 0 0 ? Z Mar24 0:00 [chrome] <defunct>
app 179 0.0 0.0 0 0 ? Z Mar24 0:00 [chrome_crashpad] <defunct>
app 184 0.0 0.0 0 0 ? Z Mar24 0:00 [chrome] <defunct>
app 185 0.0 0.0 0 0 ? Z Mar24 0:00 [chrome] <defunct>
app 202 0.0 0.0 0 0 ? Z Mar24 0:00 [chrome] <defunct>
app 285 0.0 0.0 0 0 ? Z Mar24 0:00 [chrome] <defunct>
Related
I have a problem where two of my svg have the same number of points, but something isn't right when I play the animation, the two svgs are so close together but the animation just jumps out of nowhere and it isn't right, a weird shape happens before the first svg changes to the second.
I'm doing the svgs in Adobe XD. Here is the code:
<svg id="morph" viewBox="0 0 1920 540">
<path class="morph" d="m864.216 135.95 36.39 41.917S780.519 307.11 1078.914 373.479s221.979-87.327 221.979-87.327l32.75-34.931s25.473 101.3 207.422 34.931 440.314 150.2 411.2 380.744S34.528 576.079 34.528 576.079s-3.64-429.647 342.063-509.987 272.923 174.653 487.623 69.861"/>
</svg>
<script>
var overlay = document.getElementById('morph');
var morphing = anime({
targets: '.morph',
d: [
{value : "m864.216 135.95 36.39 41.917S780.519 307.11 1078.914 373.479s221.979-87.327 221.979-87.327l32.75-34.931s25.473 101.3 207.422 34.931 440.314 150.2 411.2 380.744S34.528 576.079 34.528 576.079s-3.64-429.647 342.063-509.987 272.923 174.653 487.623 69.861"},
{value: "M2.49 576.483S20.535 398.736 122.472 239.61s236.674-199.127 302-217.883c176.407-41.244 334 45.685 334 45.685l340 233.7s172 105.427 280 119.484 322 12.3 322 12.3 118 5.271 160 61.5 56 89.613 62 117.727S2.49 576.483 2.49 576.483Z"},
],
duration: 1200,
loop: false,
easing: 'easeInOutQuint'
})
</script>
Your paths still need some optimizations to be fully compatible for interpolation.
Most animation libraries try to make paths compatible to some extent (e.g by converting them to polygons like in flubber.js).
But usually you'll get the best results cleaning up your paths manually.
step 1
step 2
m 864.216 135.95
M 2.49 576.483
l 36.39 41.917
S 20.535 398.736 122.472 239.61
S 780.519 307.11 1078.914 373.479
s 236.674 -199.127 302-217.883
s 221.979 -87.327 221.979 -87.327
c 176.407 -41.244 334 45.685 334 45.685
l 32.75 -34.931
l 340 233.7
s 25.473 101.3 207.422 34.931
s 172 105.427 280 119.484
s 440.314 150.2 411.2 380.744
s 322 12.3 322 12.3
S 34.528 576.079 34.528 576.079
s 118 5.271 160 61.5
s -3.64 -429.647 342.063-509.987
s 56 89.613 62 117.727
s 272.923 174.653 487.623 69.861
S 2.49 576.483 2.49 576.483
Z
Your command types also need to be compatible.
E.g. The second animation step has only one l (lineTo) command and a Z (closePath) missing in the first path.
Unfortunately, you can't be sure your editor/graphic app will output the same commands as it might decide to use shorthand commands (like h for horizontal lineTos )to minify the markup.
Normalize d to a reduced set of commands
This will simplify the further adjustments by converting path data to M, C, L and Z commands.
I'm using Jarek Foksa's getPathData polyfill.
path.getPathData({normalize: true});
{normalize: true} Parameter will also convert all commands to absolute coordinates.
Convert L commands to C
You can easily convert L commands to C curves by repeating the x/y coordinates like this.
L 901 178
to:
C 901 178 901 178 901 178
Adjust M starting points
Set the starting point to something like the leftmost corner/point. So your paths will be interpolated using a visual reference point.
Otherwise you might get weird flipping transitions.
Changing the M of a path is also way easier with absolute and normalized commands. You'll also find a helper function in the snippet
(1. path data chunk
=> will be appended after the second chunk)
M 864 136 (old starting point => will be deleted)
C 901 178 901 178 901 178
C 901 178 781 307 1079 373
C 1377 440 1301 286 1301 286
C 1334 251 1334 251 1334 251
C 1334 251 1359 353 1541 286
C 1723 220 1981 436 1952 667
C 1923 897 35 576 35 576 => will become the new M xy coordinate
(2. path data chunk)
C 35 576 31 146 377 66
C 722 -14 650 241 864 136
(3. path data chunk: closePath)
Z
Result:
M 35 576
C 35 576 31 146 377 66
C 722 -14 650 241 864 136
C 901 178 901 178 901 178
C 901 178 781 307 1079 373
C 1377 440 1301 286 1301 286
C 1334 251 1334 251 1334 251
C 1334 251 1359 353 1541 286
C 1723 220 1981 436 1952 667
C 1923 897 35 576 35 576
Z
Path direction
In your case both paths have a clockwise direction.
If you encounter weird flipping transitions you can try to reverse path directions.
You might use #enxaneta's great codepen example or
Svg Path commander Library.
Example 1: Normalize and change starting point (including helpers)
let svgNorm = document.querySelectorAll('.svgNorm');
svgNorm.forEach(function(svg) {
let svgPaths = svg.querySelectorAll('path');
normalizePaths(svgPaths, 0, true);
})
let orig1 = document.querySelector('.orig1');
let orig2 = document.querySelector('.orig2');
let path1 = document.querySelector('.morph1');
let path2 = document.querySelector('.morph2');
//shift starting point
shiftSvgStartingPoint(path1, 7);
//show starting points
addMarkers(orig1);
addMarkers(orig2);
addMarkers(path1);
addMarkers(path2);
function normalizePaths(paths, decimals = 1, convertLineto = false) {
paths.forEach(function(path, i) {
let pathData = path.getPathData({
normalize: true
});
pathData.forEach(function(com) {
let [type, values] = [com['type'], com['values']];
values.forEach(function(coord, c) {
com['values'][c] = +(com['values'][c]).toFixed(decimals)
})
let [x, y] = [com['values'][0], com['values'][1]];
if (type == 'L' && convertLineto) {
com['type'] = 'C';
com['values'] = [x, y, x, y, x, y];
}
})
path.setPathData(pathData)
})
}
function shiftSvgStartingPoint(path, offset) {
let pathData = path.getPathData({
normalize: true
});
let pathDataL = pathData.length;
//exclude Z/z (closepath) command if present
let lastCommand = (pathData[pathDataL - 1]['type']);
let trimR = 0;
if (lastCommand == 'Z') {
trimR = 1;
}
let newStartIndex = offset + 1 < pathData.length - 1 ? offset + 1 : pathData.length - 1 - trimR;
let newPathData = pathData;
let newPathDataL = newPathData.length;
// slice array to reorder
let newPathDataStart = newPathData.slice(newStartIndex);
let newPathDataEnd = newPathData.slice(0, newStartIndex);
// remove original M
newPathDataEnd.shift();
let newPathDataEndL = newPathDataEnd.length;
let newPathDataEndLastValues = newPathDataEnd[newPathDataEndL - 1]['values'];
let newPathDataEndLastXY = [newPathDataEndLastValues[newPathDataEndLastValues.length - 2],
newPathDataEndLastValues[newPathDataEndLastValues.length - 1]
];
//remove z(close path) from original pathdata array
if (trimR) {
newPathDataStart.pop();
newPathDataEnd.push({
'type': 'Z',
'values': []
});
}
// prepend new M command and concatenate array chunks
newPathData = [{
'type': 'M',
'values': newPathDataEndLastXY
}].concat(newPathDataStart).concat(newPathDataEnd);
// update path's d property
path.setPathData(newPathData);
return path;
}
testInterpolation(path1, path2);
function testInterpolation(path1, path2) {
path1.addEventListener('click', function(e) {
if (!path1.getAttribute('style')) {
path1.setAttribute('style', `d:path("${path2.getAttribute('d')}")`)
} else {
path1.removeAttribute('style');
}
})
}
function addMarkers(path) {
let svg = path.closest('svg');
let markerDef = document.createElementNS('http://www.w3.org/2000/svg', 'defs');
let marker =
`<marker id="circle" viewBox="0 0 10 10" refX="5" refY="5" markerWidth="10%" markerHeight="10%"
orient="auto-start-reverse">
<circle cx="5" cy="5" r="5" fill="green" />
</marker>`;
markerDef.innerHTML = marker;
svg.insertBefore(markerDef, svg.childNodes[0]);
path.setAttribute('marker-start', 'url(#circle)');
}
svg {
display: inline-block;
width: 30%;
overflow: visible;
border: 1px solid #ccc;
margin-right: 5%;
}
.row {
margin-top: 4em;
}
path {
opacity: 0.5;
transition: 0.5s;
}
<script src="https://cdn.jsdelivr.net/npm/path-data-polyfill#1.0.3/path-data-polyfill.min.js"></script>
<div>
<p>Green points illustrate starting points</p>
<svg viewBox="0 0 1920 540">
<path class="orig1" d="
m 864.216 135.95
l 36.39 41.917
S 780.519 307.11 1078.914 373.479
s 221.979 -87.327 221.979 -87.327
l 32.75 -34.931
s 25.473 101.3 207.422 34.931
s 440.314 150.2 411.2 380.744
S 34.528 576.079 34.528 576.079
s -3.64 -429.647 342.063-509.987
s 272.923 174.653 487.623 69.861
z" />
</svg>
<svg viewBox="0 0 1920 540">
<path class="orig2" d="
M 2.49 576.483
S 20.535 398.736 122.472 239.61
s 236.674 -199.127 302-217.883
c 176.407 -41.244 334 45.685 334 45.685
l 340 233.7
s 172 105.427 280 119.484
s 322 12.3 322 12.3
s 118 5.271 160 61.5
s 56 89.613 62 117.727
S 2.49 576.483 2.49 576.483
Z" />
</svg>
</div>
<div class="row">
<svg class="svgNorm" viewBox="0 0 1920 540">
<path class="morph1"
d="
m 864.216 135.95
l 36.39 41.917
S 780.519 307.11 1078.914 373.479
s 221.979 -87.327 221.979 -87.327
l 32.75 -34.931
s 25.473 101.3 207.422 34.931
s 440.314 150.2 411.2 380.744
S 34.528 576.079 34.528 576.079
s -3.64 -429.647 342.063-509.987
s 272.923 174.653 487.623 69.861
z" />
</svg>
<svg class="svgNorm" viewBox="0 0 1920 540">
<path class="morph2"
d="
M 2.49 576.483
S 20.535 398.736 122.472 239.61
s 236.674 -199.127 302-217.883
c 176.407 -41.244 334 45.685 334 45.685
l 340 233.7
s 172 105.427 280 119.484
s 322 12.3 322 12.3
s 118 5.271 160 61.5
s 56 89.613 62 117.727
S 2.49 576.483 2.49 576.483
Z" />
</svg>
<p>Click on the left path to see morphing animation. <br />Inspect this path in DevTools to get new compatible path data.</p>
</div>
Example 2: morph optimized paths with anime.js
var morphing = anime({
targets: ".morph",
d: [
{
value:
"M 2 576 C 2 576 21 399 122 240 C 224 80 359 40 424 22 C 601 -20 758 67 758 67 C 1098 301 1098 301 1098 301 C 1098 301 1270 407 1378 421 C 1486 435 1700 433 1700 433 C 1700 433 1818 438 1860 494 C 1902 551 1916 584 1922 612 C 1928 640 2 576 2 576 Z"
}
],
duration: 1200,
loop: false,
easing: "easeInOutQuint"
});
svg{
display:inline-block;
width:20em;
overflow:visible;
}
.morph{
transition:0.5s;
}
.morphPoly{
transition:0.5s;
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/animejs/3.2.1/anime.min.js"></script>
<svg id="morph" viewBox="0 0 1920 540">
<path class="morph" d="M 35 576 C 35 576 31 146 377 66 C 722 -14 650 241 864 136 C 901 178 901 178 901 178 C 901 178 781 307 1079 373 C 1377 440 1301 286 1301 286 C 1334 251 1334 251 1334 251 C 1334 251 1359 353 1541 286 C 1723 220 1981 436 1952 667 C 1923 897 35 576 35 576 Z" />
</svg>
... Quite a lot of work.
But once your paths are super compatible you can also morph between shapes via plain css. (E.g by animating/transitioning d:path() properties).
I am trying to center this wrench svg inside the rect box but I am having no luck, any thoughts. Basically Id like to center the wrench icon in the rect box.
<svg width="32" height="32" viewBox="0 0 32 32" xmlns="http://www.w3.org/2000/svg">
<rect width="32" height="32" rx="4" />
<path d="M15.0799 1.85914C14.4167 1.71925 13.7291 1.70835 13.0553 1.83113C12.0053 2.02243 11.0387 2.52916 10.284 3.28379C9.5294 4.03842 9.02267 5.00511 8.83137 6.05503C8.64007 7.10495 8.77329 8.18824 9.21327 9.16053C9.34197 9.44494 9.28105 9.77933 9.06031 10.0001L2.15031 16.9101C1.89313 17.1672 1.74866 17.516 1.74866 17.8797C1.74866 18.2434 1.89314 18.5922 2.15031 18.8494C2.40748 19.1066 2.75628 19.2511 3.11998 19.2511C3.48367 19.2511 3.83247 19.1066 4.08965 18.8494L10.9996 11.9394C11.2204 11.7187 11.5548 11.6577 11.8392 11.7864C12.8115 12.2264 13.8948 12.3596 14.9447 12.1683C15.9946 11.977 16.9613 11.4703 17.7159 10.7157C18.4706 9.96106 18.9773 8.99437 19.1686 7.94444C19.2914 7.27058 19.2805 6.58298 19.1406 5.9198L16.225 8.83537C15.8979 9.15602 15.458 9.33559 15 9.33559C14.5419 9.33559 14.1021 9.15599 13.775 8.83534L13.7696 8.83009L12.1643 7.22476C11.8437 6.89764 11.6641 6.45781 11.6641 5.99974C11.6641 5.54167 11.8437 5.10186 12.1644 4.77474L12.1696 4.76938L15.0799 1.85914ZM12.7864 0.355426C14.1363 0.109471 15.5291 0.280751 16.7792 0.846441C17.0035 0.947945 17.1637 1.15308 17.2078 1.3953C17.252 1.63752 17.1744 1.88597 17.0003 2.06007L13.2339 5.82652C13.1891 5.87306 13.1641 5.93513 13.1641 5.99974C13.1641 6.06434 13.1891 6.1264 13.2338 6.17294L14.8268 7.76588C14.8733 7.81059 14.9354 7.83559 15 7.83559C15.0646 7.83559 15.1266 7.81059 15.1732 7.76589L18.9396 3.99941C19.1137 3.82531 19.3622 3.74775 19.6044 3.79188C19.8466 3.83602 20.0518 3.99622 20.1533 4.22053C20.719 5.47062 20.8902 6.86342 20.6443 8.21332C20.3983 9.56322 19.7468 10.8061 18.7766 11.7763C17.8063 12.7466 16.5635 13.3981 15.2136 13.644C14.037 13.8584 12.8278 13.7558 11.709 13.3514L5.15031 19.9101C4.61183 20.4485 3.8815 20.7511 3.11998 20.7511C2.35846 20.7511 1.62812 20.4485 1.08965 19.9101C0.55117 19.3716 0.248657 18.6413 0.248657 17.8797C0.248657 17.1182 0.551171 16.3879 1.08965 15.8494L7.64833 9.29073C7.24389 8.17188 7.14129 6.96272 7.35567 5.78615C7.60162 4.43625 8.25313 3.19337 9.22337 2.22313C10.1936 1.25289 11.4365 0.601381 12.7864 0.355426Z" fill="#A0AEC0"/>
</svg>
One way is to use transform on the path:
<svg width="32" height="32" viewBox="0 0 32 32" xmlns="http://www.w3.org/2000/svg">
<rect width="32" height="32" rx="4" />
<path transform="translate(5, 5)" d="M15.0799 1.85914C14.4167 1.71925 13.7291 1.70835 13.0553 1.83113C12.0053 2.02243 11.0387 2.52916 10.284 3.28379C9.5294 4.03842 9.02267 5.00511 8.83137 6.05503C8.64007 7.10495 8.77329 8.18824 9.21327 9.16053C9.34197 9.44494 9.28105 9.77933 9.06031 10.0001L2.15031 16.9101C1.89313 17.1672 1.74866 17.516 1.74866 17.8797C1.74866 18.2434 1.89314 18.5922 2.15031 18.8494C2.40748 19.1066 2.75628 19.2511 3.11998 19.2511C3.48367 19.2511 3.83247 19.1066 4.08965 18.8494L10.9996 11.9394C11.2204 11.7187 11.5548 11.6577 11.8392 11.7864C12.8115 12.2264 13.8948 12.3596 14.9447 12.1683C15.9946 11.977 16.9613 11.4703 17.7159 10.7157C18.4706 9.96106 18.9773 8.99437 19.1686 7.94444C19.2914 7.27058 19.2805 6.58298 19.1406 5.9198L16.225 8.83537C15.8979 9.15602 15.458 9.33559 15 9.33559C14.5419 9.33559 14.1021 9.15599 13.775 8.83534L13.7696 8.83009L12.1643 7.22476C11.8437 6.89764 11.6641 6.45781 11.6641 5.99974C11.6641 5.54167 11.8437 5.10186 12.1644 4.77474L12.1696 4.76938L15.0799 1.85914ZM12.7864 0.355426C14.1363 0.109471 15.5291 0.280751 16.7792 0.846441C17.0035 0.947945 17.1637 1.15308 17.2078 1.3953C17.252 1.63752 17.1744 1.88597 17.0003 2.06007L13.2339 5.82652C13.1891 5.87306 13.1641 5.93513 13.1641 5.99974C13.1641 6.06434 13.1891 6.1264 13.2338 6.17294L14.8268 7.76588C14.8733 7.81059 14.9354 7.83559 15 7.83559C15.0646 7.83559 15.1266 7.81059 15.1732 7.76589L18.9396 3.99941C19.1137 3.82531 19.3622 3.74775 19.6044 3.79188C19.8466 3.83602 20.0518 3.99622 20.1533 4.22053C20.719 5.47062 20.8902 6.86342 20.6443 8.21332C20.3983 9.56322 19.7468 10.8061 18.7766 11.7763C17.8063 12.7466 16.5635 13.3981 15.2136 13.644C14.037 13.8584 12.8278 13.7558 11.709 13.3514L5.15031 19.9101C4.61183 20.4485 3.8815 20.7511 3.11998 20.7511C2.35846 20.7511 1.62812 20.4485 1.08965 19.9101C0.55117 19.3716 0.248657 18.6413 0.248657 17.8797C0.248657 17.1182 0.551171 16.3879 1.08965 15.8494L7.64833 9.29073C7.24389 8.17188 7.14129 6.96272 7.35567 5.78615C7.60162 4.43625 8.25313 3.19337 9.22337 2.22313C10.1936 1.25289 11.4365 0.601381 12.7864 0.355426Z" fill="#A0AEC0"/>
</svg>
You could also move your icon by changing the actual d path commands.
This will require to convert your path commands to relative.
This could be done e.g with Yann Armelin's SvgPathEditor
Once all comannds are relative, you just have to change the first M x/y coordinates.
<svg width="32" height="32" viewBox="0 0 32 32" xmlns="http://www.w3.org/2000/svg">
<rect width="32" height="32" rx="4" />
<path d="M20.08 6.86 c -0.66 -0.14 -1.35 -0.15 -2.02 -0.03 c -1.05 0.19 -2.02 0.7 -2.77 1.45 c -0.75 0.75 -1.26 1.72 -1.45 2.77 c -0.19 1.05 -0.06 2.13 0.38 3.11 c 0.13 0.28 0.07 0.62 -0.15 0.84 l -6.91 6.91 c -0.26 0.26 -0.4 0.61 -0.4 0.97 c 0 0.36 0.14 0.71 0.4 0.97 c 0.26 0.26 0.61 0.4 0.97 0.4 c 0.36 0 0.71 -0.14 0.97 -0.4 l 6.91 -6.91 c 0.22 -0.22 0.56 -0.28 0.84 -0.15 c 0.97 0.44 2.06 0.57 3.11 0.38 c 1.05 -0.19 2.02 -0.7 2.77 -1.45 c 0.75 -0.75 1.26 -1.72 1.45 -2.77 c 0.12 -0.67 0.11 -1.36 -0.03 -2.02 l -2.92 2.92 c -0.33 0.32 -0.77 0.5 -1.23 0.5 c -0.46 0 -0.9 -0.18 -1.22 -0.5 l -0.01 -0.01 l -1.61 -1.61 c -0.32 -0.33 -0.5 -0.77 -0.5 -1.23 c 0 -0.46 0.18 -0.9 0.5 -1.22 l 0.01 -0.01 l 2.91 -2.91 z m -2.29 -1.5 c 1.35 -0.25 2.74 -0.07 3.99 0.49 c 0.22 0.1 0.38 0.31 0.43 0.55 c 0.04 0.24 -0.03 0.49 -0.21 0.66 l -3.77 3.77 c -0.04 0.05 -0.07 0.11 -0.07 0.17 c 0 0.06 0.03 0.13 0.07 0.17 l 1.59 1.59 c 0.05 0.04 0.11 0.07 0.17 0.07 c 0.06 0 0.13 -0.02 0.17 -0.07 l 3.77 -3.77 c 0.17 -0.17 0.42 -0.25 0.66 -0.21 c 0.24 0.04 0.45 0.2 0.55 0.43 c 0.57 1.25 0.74 2.64 0.49 3.99 c -0.25 1.35 -0.9 2.59 -1.87 3.56 c -0.97 0.97 -2.21 1.62 -3.56 1.87 c -1.18 0.21 -2.39 0.11 -3.5 -0.29 l -6.56 6.56 c -0.54 0.54 -1.27 0.84 -2.03 0.84 c -0.76 0 -1.49 -0.3 -2.03 -0.84 c -0.54 -0.54 -0.84 -1.27 -0.84 -2.03 c 0 -0.76 0.3 -1.49 0.84 -2.03 l 6.56 -6.56 c -0.4 -1.12 -0.51 -2.33 -0.29 -3.5 c 0.25 -1.35 0.9 -2.59 1.87 -3.56 c 0.97 -0.97 2.21 -1.62 3.56 -1.87 z" fill="#A0AEC0"/>
</svg>
Other benefits:
usually you can decrease the svg file size using relative commands
some graphic apps might already added unnecessarily transforms (like scale(1 1), rotate(0) etc): you can keep your svg more slick and readable
Also described by Lea Verou.
I have a REST api with DRF and start to see already a performance hit with 100 objects and 1 user requesting (me - testing).
When requesting the more complex query, I get these results for CPU, always 5 - 10s:
Resource Value
>User CPU time 5987.089 msec
System CPU time 463.929 msec
Total CPU time 6451.018 msec
Elapsed time 6800.938 msec
Context switches 9 voluntary, 773 involuntary
but the SQL query stays below 100 ms
The more simple queries show similar behaviour, with CPU times around 1s and query time around 20 ms
So far, what I have tried out:
I am doing select_related() and prefetch_related(), which did improve the query time but not CPU time
I am using Imagekit to generate pictures, on a S3 instance. I removed the whole specification to test and this had minor impact
I run a method field to fetch user-specific data. Removing this had only minor impact
I have checked logs files on the backend and nothing specific shows up here...
Backend is Nginx - supervisord - gunicorn - postgresql - django 1.8.1
Here are the serializer and view:
class ParticipationOrganizationSerializer(ModelSerializer):
organization = OrganizationSerializer(required=False, read_only=True, )
bookmark = SerializerMethodField(
required=False,
read_only=True,
)
location_map = LocationMapSerializer(
required=False,
read_only=True,
)
class Meta:
model = Participation
fields = (
'id',
'slug',
'organization',
'location_map',
'map_code',
'partner',
'looking_for',
'complex_profile',
'bookmark',
'confirmed',
)
read_only_fields = (
'id',
'slug',
'organization',
'location_map',
'map_code',
'partner',
'bookmark',
'confirmed',
)
def get_bookmark(self, obj):
request = self.context.get('request', None)
if request is not None:
if(request.user.is_authenticated()):
# print(obj.bookmarks.filter(author=request.user).count())
try:
bookmark = obj.bookmarks.get(author=request.user)
# bookmark = Bookmark.objects.get(
# author=request.user,
# participation=obj,
# )
return BookmarkSerializer(bookmark).data
except Bookmark.DoesNotExist:
# We have nothing yet
return None
except Bookmark.MultipleObjectsReturned:
# This should not happen, but in case it does, delete all
# the bookmarks for safety reasons.
Bookmark.objects.filter(
author=request.user,
participation=obj,
).delete()
return None
return None
class ParticipationOrganizationViewSet(ReadOnlyModelViewSet):
"""
A readonly ViewSet for viewing participations of a certain event.
"""
serializer_class = ParticipationOrganizationSerializer
queryset = Participation.objects.all().select_related(
'location_map',
'organization',
'organization__logo_image',
).prefetch_related(
'bookmarks',
)
lookup_field = 'slug'
def get_queryset(self):
event_slug = self.kwargs['event_slug']
# Filter for the current event
# Filter to show only the confirmed participations
participations = Participation.objects.filter(
event__slug=event_slug,
confirmed=True
).select_related(
'location_map',
'organization',
'organization__logo_image',
).prefetch_related(
'bookmarks',
)
# Filter on partners? This is a parameter passed on in the url
partners = self.request.query_params.get('partners', None)
if(partners == "true"):
participations = participations.filter(partner=True)
return participations
# http://stackoverflow.com/questions/22616973/django-rest-framework-use-different-serializers-in-the-same-modelviewset
def get_serializer_class(self):
if self.action == 'list':
return ParticipationOrganizationListSerializer
if self.action == 'retrieve':
return ParticipationOrganizationSerializer
return ParticipationOrganizationListSerializer
Any help is very much appreciated!
update
I dumped the data to my local machine and I am observing similar times. I guess this rules out the whole production setup (nginx, gunicorn)?
update 2
Here are the results of the profiler.
Also I made some progress in improving the speeds by
Simplifying my serializers
Doing the tests with curl and having Debug Toolbar off
ncalls tottime percall cumtime percall filename:lineno(function)
0 0 0 profile:0(profiler)
1 0 0 3.441 3.441 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/rest_framework/views.py:442(dispatch)
1 0 0 3.441 3.441 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/rest_framework/viewsets.py:69(view)
1 0 0 3.441 3.441 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/django/views/decorators/csrf.py:57(wrapped_view)
1 0 0 3.44 3.44 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/rest_framework/mixins.py:39(list)
1 0 0 3.438 3.438 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/rest_framework/serializers.py:605(to_representation)
1 0 0 3.438 3.438 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/rest_framework/serializers.py:225(data)
1 0 0 3.438 3.438 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/rest_framework/serializers.py:672(data)
344/114 0.015 0 3.318 0.029 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/rest_framework/serializers.py:454(to_representation)
805 0.01 0 2.936 0.004 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/rest_framework/fields.py:1368(to_representation)
2767 0.013 0 2.567 0.001 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/django/dispatch/dispatcher.py:166(send)
2070 0.002 0 2.52 0.001 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/imagekit/registry.py:52(existence_required_receiver)
2070 0.005 0 2.518 0.001 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/imagekit/registry.py:55(_receive)
2070 0.004 0 2.513 0.001 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/imagekit/utils.py:147(call_strategy_method)
2070 0.002 0 2.508 0.001 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/imagekit/cachefiles/strategies.py:14(on_existence_required)
2070 0.005 0 2.506 0.001 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/imagekit/cachefiles/__init__.py:86(generate)
2070 0.002 0 2.501 0.001 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/imagekit/cachefiles/backends.py:109(generate)
2070 0.003 0 2.499 0.001 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/imagekit/cachefiles/backends.py:94(generate_now)
2070 0.01 0 2.496 0.001 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/imagekit/cachefiles/backends.py:65(get_state)
690 0.001 0 2.292 0.003 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/imagekit/cachefiles/__init__.py:148(__nonzero__)
690 0.005 0 2.291 0.003 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/imagekit/cachefiles/__init__.py:124(__bool__)
2070 0.007 0 2.276 0.001 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/imagekit/cachefiles/backends.py:112(_exists)
2070 0.01 0 2.269 0.001 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/storages/backends/s3boto.py:409(exists)
4140 0.004 0 2.14 0.001 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/storages/backends/s3boto.py:282(entries)
1633 0.003 0 2.135 0.001 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/storages/backends/s3boto.py:288()
1633 0.001 0 2.129 0.001 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/boto/s3/bucketlistresultset.py:24(bucket_lister)
2 0 0 2.128 1.064 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/boto/s3/bucket.py:390(_get_all)
2 0 0 2.128 1.064 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/boto/s3/bucket.py:426(get_all_keys)
1331 0.003 0 1.288 0.001 /usr/lib/python2.7/ssl.py:335(recv)
1331 1.285 0.001 1.285 0.001 /usr/lib/python2.7/ssl.py:254(read)
2 0 0 0.983 0.491 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/boto/connection.py:886(_mexe)
2 0 0 0.983 0.491 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/boto/s3/connection.py:643(make_request)
2 0 0 0.983 0.491 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/boto/connection.py:1062(make_request)
2 0.004 0.002 0.896 0.448 /usr/lib/python2.7/httplib.py:585(_read_chunked)
2 0 0 0.896 0.448 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/boto/connection.py:393(read)
2 0 0 0.896 0.448 /usr/lib/python2.7/httplib.py:540(read)
166 0.002 0 0.777 0.005 /usr/lib/python2.7/httplib.py:643(_safe_read)
166 0.005 0 0.775 0.005 /usr/lib/python2.7/socket.py:336(read)
2 0 0 0.568 0.284 /usr/lib/python2.7/httplib.py:793(send)
2 0 0 0.568 0.284 /usr/lib/python2.7/httplib.py:998(_send_request)
2 0 0 0.568 0.284 /usr/lib/python2.7/httplib.py:820(_send_output)
2 0 0 0.568 0.284 /usr/lib/python2.7/httplib.py:977(request)
2 0 0 0.568 0.284 /usr/lib/python2.7/httplib.py:962(endheaders)
1 0 0 0.567 0.567 /usr/lib/python2.7/httplib.py:1174(connect)
1380 0.001 0 0.547 0 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/imagekit/cachefiles/__init__.py:82(url)
1380 0.007 0 0.546 0 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/imagekit/cachefiles/__init__.py:72(_storage_attr)
105 0.009 0 0.528 0.005 /usr/lib/python2.7/socket.py:406(readline)
2 0 0 0.413 0.207 /usr/lib/python2.7/httplib.py:408(begin)
2 0 0 0.413 0.207 /usr/lib/python2.7/httplib.py:1015(getresponse)
2 0 0 0.407 0.203 /usr/lib/python2.7/httplib.py:369(_read_status)
2750 0.003 0 0.337 0 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/rest_framework/fields.py:399(get_attribute)
1 0.223 0.223 0.335 0.335 /usr/lib/python2.7/socket.py:537(create_connection)
2865 0.012 0 0.334 0 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/rest_framework/fields.py:65(get_attribute)
1610 0.005 0 0.314 0 /home/my_app/.virtualenvs/my_app/src/django-s3-folder-storage/s3_folder_storage/s3.py:13(url)
1610 0.012 0 0.309 0 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/storages/backends/s3boto.py:457(url)
690 0.005 0 0.292 0 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/imagekit/models/fields/utils.py:10(__get__)
690 0.007 0 0.251 0 /home/my_app/.virtualenvs/my_app/local/lib/python2.7/site-packages/imagekit/cachefiles/__init__.py:20(__init__)
2 0 0 0.248 0.124
>>>> cutting here, low impact calls
Looking for some expert advice here. I'm a first time sys admin on my own server and I can't figure the bottle neck in my server.
Linux CentOS 6 Apache 2.4 PHP 5.5
I've been receiving tons of high 5 min load average alert ranging between 8 - 80 from CSF
So I went ahead and installed sqltuner on my server and let it run for 3 days
The results don't show anything out of the ordinary but I'm still getting high 5 min load average daily
I'm trying to find the bottle neck (CPU, load caused by out of memory issues or I/O-bound load)
Would be stoked if someone can share any insights...
(I've included sqltuner's report and the high load email output below)
-------- Security Recommendations -------------------------------------------
[OK] There are no anonymous accounts for any database users
[OK] All database users have passwords assigned
[!!] There is no basic password file list!
-------- Performance Metrics -------------------------------------------------
[--] Up for: 120d 18h 27m 20s (227M q [21.795 qps], 51M conn, TX: 907B, RX: 26B)
[--] Reads / Writes: 38% / 62%
[--] Binary logging is disabled
[--] Total buffers: 15.4G global + 4.1M per thread (600 max threads)
[OK] Maximum reached memory usage: 16.2G (51.75% of installed RAM)
[OK] Maximum possible memory usage: 17.8G (56.91% of installed RAM)
[OK] Slow queries: 0% (11/227M)
[OK] Highest usage of available connections: 33% (199/600)
[OK] Aborted connections: 0.56% (284327/51183230)
[OK] Query cache efficiency: 83.0% (78M cached / 94M selects)
[!!] Query cache prunes per day: 10288
[OK] Sorts requiring temporary tables: 0% (392 temp sorts / 1M sorts)
[OK] Temporary tables created on disk: 4% (65K on disk / 1M total)
[OK] Thread cache hit rate: 99% (199 created / 51M connections)
[OK] Table cache hit rate: 22% (425 open / 1K opened)
[OK] Open file limit used: 0% (433/50K)
[OK] Table locks acquired immediately: 99% (41M immediate / 41M locks)
-------- MyISAM Metrics -----------------------------------------------------
[!!] Key buffer used: 20.2% (108M used / 536M cache)
[OK] Key buffer size / total MyISAM indexes: 512.0M/14.7M
[OK] Read Key buffer hit rate: 99.8% (51M cached / 121K reads)
[!!] Write Key buffer hit rate: 40.8% (4M cached / 2M writes)
-------- InnoDB Metrics -----------------------------------------------------
[--] InnoDB is enabled.
[OK] InnoDB buffer pool / data size: 14.6G/140.9M
[!!] InnoDB buffer pool instances: 1
[!!] InnoDB Used buffer: 3.39% (32546 used/ 959999 total)
[OK] InnoDB Read buffer efficiency: 100.00% (5437258684 hits/ 5437259670 total)
[!!] InnoDB Write buffer efficiency: 0.00% (0 hits/ 1 total)
[OK] InnoDB log waits: 0.00% (0 waits / 24069213 writes)
-------- AriaDB Metrics -----------------------------------------------------
[--] AriaDB is disabled.
-------- Replication Metrics -------------------------------------------------
[--] No replication slave(s) for this server.
[--] This is a standalone server..
-------- Recommendations -----------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
Increasing the query_cache size over 128M may reduce performance
Variables to adjust:
query_cache_size (> 128M) [see warning above]
innodb_buffer_pool_instances(=14)
----------------------
(The only change I've made is to reduce InnoDB size and add multiple pool instances)
The high daily load email:
Time: Sun Dec 6 05:43:53 2015 -0500
1 Min Load Avg: 80.26
5 Min Load Avg: 21.19
15 Min Load Avg: 7.46
Running/Total Processes: 221/875
ps.txt
O
Scoreboard Key:
"_" Waiting for Connection, "S" Starting up, "R" Reading Request,
"W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup,
"C" Closing connection, "L" Logging, "G" Gracefully finishing,
"I" Idle cleanup of worker, "." Open slot with no current process
Srv PID Acc M CPU SS Req Conn Child Slot Client VHost Request
0-1424 16243 0/149/2713129 W 1.28 14 0 0.0 2.34 65107.59 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
1-1424 17057 0/18/2701770 W 2.15 4 0 0.0 0.30 62402.50 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
2-1424 17064 0/24/2685073 W 2.11 13 0 0.0 0.32 62668.14 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
3-1424 15319 0/215/2657841 W 3.50 4 0 0.0 3.88 61950.21 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
4-1424 11567 0/204/2651294 W 7.10 7 0 0.0 3.00 63562.61 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
5-1424 16512 0/37/2640191 W 2.19 5 0 0.0 0.60 63637.48 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
6-1424 17735 0/8/2630311 W 0.62 19 0 0.0 0.06 65036.68 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
7-1424 16521 0/31/2613938 W 2.20 19 0 0.0 0.36 62385.07 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
8-1424 16081 0/33/2611913 W 2.46 5 0 0.0 0.42 60535.12 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
9-1424 14711 0/120/2603042 W 1.89 18 0 0.0 2.11 59868.26 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
10-1424 16838 0/21/2592501 W 1.77 15 0 0.0 0.24 62195.33 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
11-1424 16531 0/42/2584776 W 2.45 11 0 0.0 0.39 62253.11 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
12-1424 17065 0/20/2570161 W 1.29 12 0 0.0 0.18 60474.65 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
13-1424 17770 0/13/2564128 W 1.27 2 0 0.0 0.63 59748.24 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
14-1424 17771 0/14/2542936 W 1.30 2 0 0.0 0.17 60513.73 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
15-1424 15736 0/64/2536855 W 2.91 7 0 0.0 1.16 61453.61 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
16-1424 17077 0/19/2522131 W 2.76 15 0 0.0 0.35 59307.60 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
17-1424 14723 0/93/2521068 W 3.38 6 0 0.0 1.77 60437.40 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
18-1424 16279 0/62/2509938 W 1.81 15 0 0.0 1.07 61401.24 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
19-1424 15333 0/116/2498356 W 3.24 19 0 0.0 1.69 57911.45 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
20-1424 16297 1/35/2494463 W 0.98 53 62 16.1 0.47 59474.66 58.174.24.65 suspensionrevolution.com:80 GET /new/wp-content/themes/optimizePressTheme/lib/assets/defaul
21-1424 16298 0/40/2473943 W 3.83 3 0 0.0 0.54 57987.71 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
22-1424 18054 0/6/2469193 W 1.23 1 0 0.0 0.05 59122.65 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
23-1424 12894 0/162/2458774 W 5.90 17 0 0.0 2.42 56404.92 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
24-1424 18088 0/4/2452422 W 0.90 11 0 0.0 0.00 58405.08 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
25-1424 18089 0/6/2446570 W 1.22 1 0 0.0 0.03 57036.34 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
26-1424 17079 0/30/2439491 W 1.88 0 0 0.0 0.43 54697.67 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
27-1424 16101 0/64/2416961 W 1.53 18 0 0.0 1.69 57160.43 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
28-1424 18140 0/9/2403931 W 0.62 18 0 0.0 0.02 55901.03 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
29-982 1505 1/18/1548355 G 0.14 2914733 450294 2.8 0.29 34947.04 96.47.70.4 suspensionrevolution.com:80 POST /dap/dap-clickbank-6.0.php HTTP/1.1
30-1424 15338 0/100/2384316 W 2.20 7 0 0.0 1.24 53919.77 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
31-1424 16300 2/56/2379897 K 3.47 3 1365 2.4 1.01 55195.75 89.166.18.35 appcoiner.com:80 GET /favicon.ico HTTP/1.1
32-1424 15749 0/108/2369131 W 3.26 17 0 0.0 2.09 55452.02 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
33-1424 17100 0/17/2359616 W 1.70 11 0 0.0 0.16 52564.23 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
34-1424 16310 0/162/2356424 W 3.95 15 0 0.0 2.68 55800.32 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
35-1424 16543 0/63/2326471 W 1.29 4 0 0.0 0.75 55028.80 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
36-1424 17101 0/18/2331624 W 2.05 14 0 0.0 0.21 53656.66 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
37-1424 17102 0/20/2314444 W 1.51 19 0 0.0 0.29 55684.29 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
38-1424 19665 0/1/2295814 W 0.00 3 0 0.0 0.00 52187.61 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
39-984 8727 1/69/1464486 G 0.68 2906097 450284 2.8 0.83 33844.50 74.63.153.4 suspensionrevolution.com:80 POST /dap/dap-clickbank-6.0.php HTTP/1.1
40-1424 19720 0/1/2277467 W 0.00 2 0 0.0 0.00 55864.93 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
41-1424 18141 0/6/2270838 W 0.62 14 0 0.0 0.02 54059.95 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
42-983 18177 1/49/1440183 G 0.63 2910665 450302 2.8 0.57 31224.74 74.63.153.4 suspensionrevolution.com:80 POST /dap/dap-clickbank-6.0.php HTTP/1.1
43-1424 16104 2/57/2242969 W 2.62 5 0 8.8 0.83 56170.39 54.202.7.147 appcoiner.com:80 GET /start-2/?utm_expid=111102625-1.-ThtNpCTSByWcbkMGdBOow.1&ho
44-1424 16547 0/28/2247277 W 3.84 5 0 0.0 0.31 53028.08 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
45-1424 15797 0/80/2225028 W 3.24 5 0 0.0 1.63 51333.94 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
46-1424 19721 0/1/2205346 W 0.00 2 0 0.0 0.00 52025.79 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
47-1424 18142 0/11/2207016 W 0.94 1 0 0.0 0.07 51355.07 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
48-1424 17104 0/137/2172322 W 1.28 7 0 0.0 2.32 49665.11 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
49-1424 16314 0/63/2168481 W 4.14 5 0 0.0 1.16 49191.04 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
50-1424 19763 0/0/2141243 W 2.41 12 0 0.0 0.00 49538.97 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
51-1424 17106 0/20/2137681 W 1.24 7 0 0.0 0.29 49973.70 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
52-1424 16549 0/34/2125106 W 1.99 7 0 0.0 0.50 50442.63 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
53-1424 17107 0/20/2109740 W 2.18 2 0 0.0 0.31 48074.92 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
54-1424 18143 0/4/2087977 W 1.21 8 0 0.0 0.00 49243.64 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
55-1424 17114 0/30/2062106 W 0.34 17 0 0.0 0.46 48605.36 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
56-1424 17115 0/19/2064562 W 2.07 0 0 0.0 0.30 47600.46 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
57-1424 16569 0/41/2051051 W 3.50 8 0 0.0 0.54 47547.57 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
58-1424 17116 0/28/2023150 W 1.28 4 0 0.0 0.42 49170.14 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
59-1424 17117 0/30/2010767 W 1.41 1 0 0.0 0.51 47681.03 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
60-1424 17118 0/17/1999913 R 0.05 53 5 0.0 0.26 46914.84 65.30.135.196
61-1424 16572 0/35/1978028 W 2.73 16 0 0.0 0.45 45848.82 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
62-1424 16573 0/27/1957566 W 1.60 0 0 0.0 0.45 46768.01 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
63-1424 16574 0/43/1936669 W 2.37 3 0 0.0 0.54 43520.07 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
64-1424 16575 0/28/1922381 W 1.54 1 0 0.0 0.33 45007.49 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
65-1424 16576 0/32/1903916 W 2.15 13 0 0.0 0.73 45117.83 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
66-1424 17119 0/28/1878566 W 1.18 6 0 0.0 0.45 44448.25 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
67-1424 16578 1/39/1869043 K 2.01 0 0 1.2 0.53 44966.25 114.79.47.51 suspensionrevolution.com:80 GET /favicon.ico HTTP/1.1
68-1424 16579 0/37/1841958 W 2.59 5 0 0.0 0.48 44262.26 72.5.231.11 appcoiner.com:80 GET /?hopc2s=nakt123 HTTP/1.1
This is my TOP output:
root#ns513521 [~]# top
top - 08:21:31 up 153 days, 3:51, 1 user, load average: 0.15, 0.27, 0.51
Tasks: 230 total, 2 running, 227 sleeping, 0 stopped, 1 zombie
Cpu(s): 0.4%us, 0.3%sy, 0.0%ni, 99.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 32855908k total, 25102984k used, 7752924k free, 886004k buffers
Swap: 1569780k total, 63984k used, 1505796k free, 21254784k cached
This is my ioStat output:
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 21.99 1476.57 549.50 19540299168 7271856568
sdb 19.53 982.85 549.49 13006647390 7271718616
sdc 19.46 978.26 549.49 12945853934 7271718616
md2 9.78 492.22 399.94 6513868322 5292584264
md1 20.05 27.80 140.15 367920858 1854711136
I think you can just use sar, e.g. something like this :
sar -q -s 00:00:00 -e 11:59:59 -f /var/log/sa/sa`date +%d | awk '{printf "%02d", $1 - 1}'`
I am running a EC2 small instance as my production server. It has 1.7G memory. I noticed it uses almost all memory. However when I check top output, it looks like that only 30% is actually used. Did I misread the top output?
Here is the top output (sorted by %MEM)
top - 21:33:15 up 141 days, 9:39, 2 users, load average: 0.00, 0.00, 0.00
Tasks: 81 total, 2 running, 79 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 1747660k total, 1733580k used, 14080k free, 224144k buffers
Swap: 917496k total, 132k used, 917364k free, 1144808k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
11664 mysql 15 0 794m 83m 5020 S 0.0 4.9 0:17.34 mysqld
12845 nobody 25 0 52416 38m 3200 S 0.0 2.3 0:02.10 ruby1.8
12847 nobody 16 0 52704 38m 2068 S 0.0 2.2 0:02.08 ruby1.8
12023 www-data 15 0 37692 10m 4164 S 0.0 0.6 0:01.28 apache2
11979 www-data 15 0 37660 10m 4172 S 0.0 0.6 0:01.24 apache2
12020 www-data 15 0 37708 10m 4120 S 0.0 0.6 0:01.17 apache2
12263 www-data 15 0 37708 10m 4176 S 0.0 0.6 0:00.83 apache2
11989 www-data 15 0 37720 10m 4024 S 0.0 0.6 0:01.28 apache2
12014 www-data 15 0 37468 10m 4172 S 0.0 0.6 0:01.17 apache2
12021 www-data 15 0 37652 10m 3992 S 0.0 0.6 0:01.25 apache2
12054 www-data 15 0 37480 10m 4176 S 0.0 0.6 0:01.33 apache2
11990 www-data 15 0 37448 10m 4188 S 0.0 0.6 0:01.16 apache2
12024 www-data 16 0 37416 10m 4172 S 0.0 0.6 0:01.00 apache2
11991 www-data 15 0 37432 10m 4148 S 0.0 0.6 0:01.24 apache2
11984 www-data 15 0 37444 9.8m 3972 S 0.0 0.6 0:01.33 apache2
11985 www-data 15 0 37444 9.8m 3948 S 0.0 0.6 0:01.18 apache2
11982 www-data 15 0 37408 9.8m 3968 S 0.0 0.6 0:01.12 apache2
12013 www-data 17 0 37432 9.8m 4152 S 0.0 0.6 0:01.19 apache2
12052 www-data 15 0 37176 9.8m 4180 S 0.0 0.6 0:01.29 apache2
11981 www-data 15 0 37172 9.8m 4168 S 0.0 0.6 0:01.40 apache2
12395 www-data 15 0 37420 9988 3972 S 0.0 0.6 0:00.72 apache2
12015 www-data 15 0 37412 9972 3900 S 0.0 0.6 0:01.31 apache2
11987 www-data 15 0 37160 9956 4136 S 0.0 0.6 0:01.22 apache2
12022 www-data 15 0 37140 9900 4140 S 0.0 0.6 0:01.20 apache2
12051 www-data 15 0 37216 9848 3976 S 0.0 0.6 0:01.31 apache2
11978 www-data 18 0 36948 9784 4180 S 0.0 0.6 0:01.08 apache2
11975 www-data 15 0 37140 9772 3972 S 0.0 0.6 0:01.49 apache2
12019 www-data 15 0 37148 9752 3944 S 0.0 0.6 0:01.08 apache2
11970 www-data 15 0 36920 9736 4160 S 0.0 0.6 0:01.25 apache2
11974 www-data 15 0 36848 9656 4148 S 0.0 0.6 0:01.53 apache2
11973 www-data 15 0 36924 9552 3972 S 0.0 0.5 0:01.19 apache2
28622 root 18 0 35232 9232 5592 S 0.0 0.5 0:00.30 apache2
11969 www-data 15 0 36340 9132 4136 S 0.0 0.5 0:01.51 apache2
12018 www-data 19 0 36332 9124 4136 S 0.0 0.5 0:01.32 apache2
11972 www-data 15 0 36320 8968 3988 S 0.0 0.5 0:01.33 apache2
12012 www-data 15 0 35796 8600 4144 S 0.0 0.5 0:01.11 apache2
11965 root 15 0 17356 7552 1644 S 0.0 0.4 0:00.13 ruby1.8
12848 root 15 0 8384 2744 2164 R 0.0 0.2 0:00.12 sshd
12762 root 15 0 8384 2724 2164 S 0.0 0.2 0:00.01 sshd
11302 postfix 18 0 6184 2576 1880 S 0.0 0.1 0:00.02 tlsmgr
11964 root 16 0 8188 2248 1492 S 0.0 0.1 0:00.06 ApplicationPool
23997 postfix 22 0 5856 1852 1488 S 0.0 0.1 0:00.22 qmgr
12850 root 15 0 4408 1848 1436 S 0.0 0.1 0:00.00 bash
12764 root 25 0 4396 1800 1400 S 0.0 0.1 0:00.00 bash
23996 root 15 0 5804 1780 1428 S 0.0 0.1 0:01.01 master
13036 postfix 17 0 5812 1684 1356 S 0.0 0.1 0:00.00 pickup
1051 klog 18 0 2884 1676 436 S 0.0 0.1 0:00.04 klogd
13035 root 15 0 2468 1164 916 R 0.0 0.1 0:00.01 top
5841 nobody 15 0 2652 1120 684 S 0.0 0.1 0:00.50 memcached
11509 root 15 0 5456 1068 676 S 0.0 0.1 0:00.00 sshd
1163 root 18 0 3560 1060 872 S 0.0 0.1 0:01.46 cron
1 root 18 0 2032 840 580 S 0.0 0.0 0:04.20 init
4070 syslog 18 0 2056 732 568 S 0.0 0.0 7:25.48 syslogd
908 root 16 -2 2292 656 528 S 0.0 0.0 0:00.06 dhclient3
The 'used' count includes filesystem cache and kernel buffers. The cached memory can be free'd when an application requires more heap. You are right to say that only about 30% is actually used, since 65% of that is cache, and 12% is buffers.
The kernel will release the cached memory when an application attempts to allocate more memory, this is normal behavior and I see no problem with your memory usage.
When you use significant amounts of swap, and your 'cached' count is very low - then you have a problem.
Some additional helpful information here (applicable to any Linux distro) -
http://forums.gentoo.org/viewtopic.php?t=175419
Mem: 1747660k total, 1733580k used, 14080k free, 224144k buffers
compare the total and used :-)
It is used for file buffering. It is nothing wrong since good memory managment should always use all availble memory in system. I don't remember but I think that 1144808k cached is the memory you can't find.
You can try writing simple application that reserves about 1 GB of memory and release it and quit. Then probably you should have this 1 GB counted as free memory since file buffers was removed.