I've been having a hard time debugging the memory leak.
I have paging using HTTParty inside Sidekiq, and the memory is keep growing and growing. I profiled the sidekiq and this is what I found:
540 /home/user/.rvm/gems/ruby-2.2.3/gems/aws-sdk-core-2.4.2/lib/seahorse/client/configuration.rb:157:DATA
540 /home/user/.rvm/gems/ruby-2.2.3/gems/aws-sdk-core-2.4.2/lib/seahorse/client/configuration.rb:157:NODE
609 /home/user/.rvm/gems/ruby-2.2.3/gems/activesupport-4.1.14/lib/active_support/core_ext/class/attribute.rb:86:DATA
1376 /home/user/.rvm/gems/ruby-2.2.3/gems/activerecord-4.1.14/lib/active_record/connection_adapters/postgresql/database_statements.rb:148:STRING
1712 /home/user/.rvm/gems/ruby-2.2.3/gems/activesupport-4.1.14/lib/active_support/dependencies.rb:247:ARRAY
1713 /home/user/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/set.rb:290:STRING
1964 /home/user/.rvm/gems/ruby-2.2.3/gems/mime-types-3.1/lib/mime/types/_columnar.rb:29:OBJECT
1964 /home/user/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/set.rb:72:OBJECT
2572 /home/user/.rvm/gems/ruby-2.2.3/gems/mime-types-3.1/lib/mime/types/_columnar.rb:24:STRING
2987 /home/user/.rvm/gems/ruby-2.2.3/gems/activesupport-4.1.14/lib/active_support/dependencies.rb:247:NODE
3126 /home/user/.rvm/gems/ruby-2.2.3/gems/mime-types-3.1/lib/mime/types/container.rb:10:OBJECT
3506 /home/user/.rvm/gems/ruby-2.2.3/gems/activesupport-4.1.14/lib/active_support/dependencies.rb:247:DATA
3928 /home/user/.rvm/gems/ruby-2.2.3/gems/mime-types-3.1/lib/mime/type.rb:532:STRING
3928 /home/user/.rvm/gems/ruby-2.2.3/gems/mime-types-3.1/lib/mime/type.rb:543:STRING
5175 /home/user/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/set.rb:81:HASH
5234 /home/user/.rvm/gems/ruby-2.2.3/gems/activesupport-4.1.14/lib/active_support/dependencies.rb:247:STRING
5892 /home/user/.rvm/gems/ruby-2.2.3/gems/mime-types-3.1/lib/mime/type.rb:546:STRING
8120 /home/user/.rvm/gems/ruby-2.2.3/gems/json-1.8.3/lib/json/common.rb:155:ARRAY
87403 /home/user/.rvm/gems/ruby-2.2.3/gems/json-1.8.3/lib/json/common.rb:155:HASH
541817 /home/user/.rvm/gems/ruby-2.2.3/gems/json-1.8.3/lib/json/common.rb:155:STRING
Notice the json STRING and HASH. These bottom 2 just keeps growing as the sidekiq keeps on running more and more jobs.
The code is this:
...
begin
reactions = HTTParty.get("https://graph.facebook.com/v2.7/#{vid.external_id}/reactions?summary=true&limit=#{PAGE_SIZE}&access_token=#{ENV['at']}")
rescue Exception => ex
return
end
r_last_id = "vl/programs/#{vid.program_id}/vids/#{vid.id}/reactions/last_id"
r_last_entry_idx = "vl/programs/#{vid.program_id}/vids/#{vid.id}/reactions/last_entry_idx"
reactions_purged = []
abort = false
total_records = reactions['summary']['total_count'] || 0
last_total_count = $redis.get(r_last_entry_idx).to_i
need_to_run = total_records - last_total_count
need_to_run = 0 if need_to_run < 0
last_id = nil
loop do
break if reactions['data'].nil? || reactions['data'].empty?
reactions['data'].each do |r|
last_id = $redis.get(r_last_id)
abort = true and break if !last_id.nil? && need_to_run <= 0
need_to_run -= 1
reactions_purged << r
end
GC.start # <----- Even this didnt solve it
break if abort
break if reactions['paging']['next'].nil?
reactions = HTTParty.get(reactions['paging']['next'])
end
...
Even the GC.start I added as you see, didn't solve it.
What causes this JSON leakage? this is coming from HTTParty...
Thanks
Related
I'm making an action RPG in Love2d, and I moved all of my player code into a separate Lua script for the sake of organization - but now upon attempting to use player.load, I get this error:
Error
main.lua:22: attempt to index global 'player' (a boolean value)
Traceback
main.lua:22: in function 'load'
[C]: in function 'xpcall'
[C]: in function 'xpcall'
this is my main.lua script:
-- RPG PROJECT IN LOVE2D
-- debug mode
debug = true -- SET FALSE BEFORE SHIPPING
-- ROOM CHART:
-- 0 - Title
-- 1 - Overworld
-- 2 - Shop
-- 3 - Boss Room
Room = 0
-- PLAYER STATE CHART:
-- 0 - free
-- 1 - attacking
-- 3 - can't move
function love.load(dt)
player = require 'Scripts/player'
player.load()
sti = require 'Libraries/sti'
gameMap = sti('Maps/overworld.lua')
menuImg = love.graphics.newImage('Assets/Menu.png')
love.window.setMode(800, 600)
end
function love.draw(dt)
if Room == 0 then
love.graphics.draw(menuImg, 0, 0)
end
if Room == 1 then
gameMap:draw()
player.draw()
end
if debug == true then
love.graphics.print("Current FPS: "..tostring(love.timer.getFPS( )), 10, 10)
end
end
function love.update(dt)
if Room == 0 then
if love.keyboard.isDown('space') then
Room = 1
end
end
if Room == 1 then
player.Update(dt)
if love.keyboard.isDown('escape') then
Room = 0
end
end
if love.keyboard.isDown('t') then
debug = not debug
end
end
and this is my player.lua script:
-- PLAYER SCRIPT
player = {x = 50, y = 50, speed = 3, state = 0, img = nil}
function player.load()
playerRight = love.graphics.newImage('Assets/playerRight.png')
playerLeft = love.graphics.newImage('Assets/playerLeft.png')
playerUp = love.graphics.newImage('Assets/playerUp.png')
playerDown = love.graphics.newImage('Assets/playerDown.png')
player.img = playerDown
end
function GetInput()
if player.state == 0 or player.state == 1 then
if love.keyboard.isDown('w') then
player.y = player.y - player.speed
player.img = playerUp
elseif love.keyboard.isDown('s') then
player.y = player.y + player.speed
player.img = playerDown
end
if love.keyboard.isDown('a') then
player.x = player.x - player.speed
player.img = playerLeft
elseif love.keyboard.isDown('d') then
player.x = player.x + player.speed
player.img = playerRight
end
end
end
function player.update(dt)
GetInput()
end
function player.draw()
love.graphics.draw(player.img, player.x, player.y)
end
Any help would be much appreciated!
Here's my folders as well, just in case it's a path issue:
UPDATE:
I solved it by renaming the player object to oPlayer, that was what was giving me an error.
I have severall Hints for you
Unclear
It looks you write it under Linux but should it also run under Windows?
If so, use OS independent folder delimiter ( the dot ) in require().
Example
player = require 'Scripts.player'
In General: Dont use Slash / Backslash \ in require()
Tip: LÖVE uses LuaJIT with modified Lua 5.1 check the Windows OS is easy
(For deciding to use Slash or Backslash (not in/for require()))
Clear
As #Luke100000 mentioned...
A Script for require has to return something.
If not than only a true wil be cached and a next require returns only true.
Therefore your player.lua content should be...
-- player.lua
local player = {x = 50, y = 50, speed = 3, state = 0, img = nil}
-- player.load() will be local
-- GetInput() wil be global
-- And the last line...
return(player)
I'm trying to get my save_game() method to work, nothing is getting written to the JSON file. This is my first assignment working with JSON and serialization in general. I'm not quite sure where I'm even going wrong.
These are my serialization methods:
def to_json
JSON.generate({array: #array, filestuff: #filestuff, random_word: #random_word, cipher: #cipher, random_word2: #random_word2, counter: #counter})
end
def load
game_file = File.read("saved.json")
data = JSON.parse(game_file)
#cipher = data["cipher"]
#random_word2 = data["random_word2"]
#counter = data["counter"]
end
def save_game(string)
game_file = File.new("saved.json","w")
game_file.write(string)
game_file.close
end
This is my program, on line 92 I try to call my save_game method.
require 'json'
load 'display.rb'
class Hangman
attr_accessor :name
#name = name
def initialize
puts "What is your name?"
#name = gets.chomp
puts "
################################################
HANGMAN
################################################
_________
|
| |
| O
| /|\\
| |
| / \\
|
-----------------
Welcome #{#name} to Hangman. The computer will generate
a 5-12 letter random word. You will try to guess
that word one letter at a time. Try to solve the
puzzle before time runs out!
"
end
end
class Gameplay
attr_accessor :array, :filestuff, :random_word, :cipher, :random_word2, :counter
def initialize
#array = []
#filestuff = File.foreach('5text.txt') do |x|
chomped = x.chomp
#array << chomped if (chomped.length >= 5 and chomped.length <= 12)
end
#random_word = #array.sample
#cipher = #random_word.gsub(/[a-z]/, '*').split(//)
#random_word2 = #random_word.split(//)
#counter = 5
def to_json
JSON.generate({array: #array, filestuff: #filestuff, random_word: #random_word, cipher: #cipher, random_word2: #random_word2, counter: #counter})
end
def load
game_file = File.read("saved.json")
data = JSON.parse(game_file)
#cipher = data["cipher"]
#random_word2 = data["random_word2"]
#counter = data["counter"]
end
def save_game(string)
game_file = File.new("saved.json","w")
game_file.write(string)
game_file.close
end
def choice(n)
#random_word2.each_with_index do |i,index|
if i == n
#cipher[index] = i
end
end
if n == #random_word2.join.to_s
puts "You win"
puts "would you like to start another game? Y/N"
new_game = gets.chomp
if new_game == "Y"
Hangman.new
else exit
end
end
if #random_word2.include?(n) == false
#counter -= 1
display
puts "#{#counter} guesses remaining."
puts "To save press 1"
save = gets.chomp
if save == "1"
#Will not save
save_game($b.to_json)
end
end
if #counter == 0
puts "would you like to start another game? Y/N"
new_game = gets.chomp
if new_game == "Y"
else exit
end
end
puts #cipher.join
end
#counter = 5
while #counter > 0
choice(gets.chomp)
end
end
end
Hangman.new
$b = Gameplay.new
You need to close the file in order to make sure your output is actually written to the disk ("flushed"). You can manually, call close:
def save_game(string)
game_file = File.new("saved.json","w")
game_file.write(string)
game_file.close
end
or, you can use File.open, which takes a block and closes the file when the block ends:
File.open("saved.json", "w") do |game_file|
game_file.write(string)
end
Since, writing to the disk is a slow operation, Ruby (and all languages that I can think of right now) will hold off on actually writing the file until it has accumulated a certain amount of text in a buffer. Once it has reached this limit, it will flush the buffer and write everything in it to disk. In order to make sure all your text is actually written when trying to write a file, you need to call close on the file, and as part of closing it, Ruby will flush whatever is left in its buffer.
There are other ways of making sure your content is flushed but when you're just starting to learn about this stuff, it should suffice to just make sure to always close files when you're done reading or writing them.
I want to the mysql-proxy lua script to handle interleaving accesses to a website (e.g. two different browser windows/users) but being able to pause/delay one of the two without influencing the other.Handling sessions interleavingly is possible in mysql-proyx lua (so it seems regarding the later listed output) but as soon as I start delaying the script it blocks everything and the other session cannot advance either.
-- the query which indicates the session/connection that shall be delayed at that execution
local qoi = "SELECT loginattempts,uid FROM mybb_users WHERE username='user1' LIMIT 1"
function read_query(packet)
if string.byte(packet) == proxy.COM_QUERY then
query = packet:sub(2)
start_time = os.time()
if query == qoi then
print("busy wait")
while os.time() < start_time + 20 do
--nothing
end
print("busy wait end")
end
print("Connection id: " .. proxy.connection.server.thread_id)
end
end
However this script ends up with output:
Connection id: 36
busy wait
busy wait end
Connection id: 36
Connection id: 36
Connection id: 36
Connection id: 37
Connection id: 37
Connection id: 36
Connection id: 36
Connection id: 36
Connection id: 37
and not the expected
Connection id: 36
busy wait
connection id: 37
connection id: 37
busy wait end
Connection id: 36
Is my intention even achievable and if so how?
It seems to be impossible to delay the session within lua but it works just as fine if I outsource the delay to mysql server as this will force the interleaving as well.
local DEBUG = true
local qoi = "SELECT loginattempts,uid FROM mybb_users WHERE username='user1' LIMIT 1"
function read_query(packet)
ret = nil
comp_query = qoi
if string.byte(packet) == proxy.COM_QUERY then
query = packet:sub(2)
if query == comp_query then
if DEBUG then
print("found matching query " .. packet:sub(2))
print("insert sleep")
end
inj_query = "SELECT sleep(30);"
new_packet = string.char(proxy.COM_QUERY) .. inj_query
proxy.queries:append(1, new_packet, { resultset_is_needed = true })
proxy.queries:append(2, packet, { resultset_is_needed = true })
ret = proxy.PROXY_SEND_QUERY
end
end
return ret
end
function read_query_result(inj)
if inj.id == 1 then
if DEBUG then
print("sleep query returns")
end
return proxy.PROXY_IGNORE_RESULT
end
if inj.id == 2 then
if DEBUG then
print("regular query returns")
end
return
end
return
end
In my application I have positions. Each position has one parent and many children positions. How can I navigate the tree and find all positions that are in the same generation?
Currently I have this:
def org_chart_level
ps = self
level = 10
while ps do
break if ps.reports_to == nil
ps = ps.reports_to
level += 10
end
if self.reports_to
siblings = self.reports_to.responsible_for
else
siblings = [self]
puts siblings
end
"#{level}-#{siblings.index(self) + 1}"
end
This almost works because it can give me the level of a position and it can tell me which sibling it is of the parent, but it cannot tell me which sibling it is in the generation.
Expected level for each position.
1.1
-2.1
-3.1
-2.2
-3.2
Actual level:
1.1
-2.1
-3.1
-2.2
-3.1
I ended up storing the positions generation in the database before it is saved.
before_save :find_generation
def find_generation
ps = self
i = 0
while ps do
break if ps.reports_to == nil
ps = ps.reports_to
i += 1
end
self.generation = i
end
And then when I need to generate the org chart level:
def org_chart_level
ps = self
level = 10
while ps do
break if ps.reports_to == nil
ps = ps.reports_to
level += 10
end
generation = organization.positions.where(generation: self.generation) #position belongs to organization.
"#{level}-#{generation.index(self) + 1}"
end
i have two images on the screen that overlap called A and B. i have a third image called C that the user is able to move around the screen. the bigger one (A) of the two resets the C to the spawn point on collide. i need to know how to make it so that C can walk over image B and not be reset to the spawn point. as in image B overlaps image A and negates the collision function.
here is the code:
--Hides status bar from top of page
display.setStatusBar( display.HiddenStatusBar )
--Set a test background image
local backgroundimg = display.newImageRect("testbackground.png", 320,576)
backgroundimg.x = display.contentWidth*0.5
backgroundimg.y = display.contentHeight*0.5
--Set the position and amount of score and lives
local score = 0
local lives = 5
local showscore = display.newText("Score: "..score,0,-36,native.systemFont,25)
showscore:setTextColor(0, 0, 0)
local showlives = display.newText("Lives: "..lives,230,-36,native.systemFont,25)
showlives:setTextColor(0, 0, 0)
--Set physics for collisions, etc.
physics = require("physics")
physics.start()
physics.setGravity(0,0)
--set water
local water = display.newImageRect("water.png",320,192)
water.x = display.contentWidth*0.5
water.y = 144
physics.addBody(water,"static")
water:addEventListener("collision", function()timer.performWithDelay(50,waterCollide)end)
function waterCollide(event)
lives = lives - 1
display.remove(frog)
frog = display.newImageRect("FrogTest.png",32,48)
frog.x = display.contentWidth*0.5
frog.y = 504
physics.addBody(frog, "dynamic")
frog.isFixedRotation = true
end
--Sets buttons images and positions
local forward = display.newImageRect("Forward Button.png",106,100)
forward.x = 160
forward.y = 478
local left = display.newImageRect("Left Button.png",106,100)
left.x = 53
left.y = 478
local right = display.newImageRect("Right Button.png",106,100)
right.x = 267
right.y = 478
--Set log position and movement
local log1 = display.newImageRect("log1.png", 96, 48)
log1.x = 32
log1.y = 226
physics.addBody(log1,"kinematic")
transition.to(log1, {time = 3500, x = 288})
--Set a frog sprite on the screen
frog = display.newImageRect("FrogTest.png",32,48)
frog.x = display.contentWidth*0.5
frog.y = 504
physics.addBody(frog, "dynamic",{density = 1.0, friction = 1, bounce = -1})
frog.isFixedRotation = true
--Sets motion variables
local motionX = 0
local motionY = 0
local speed = 4
--Moving forward
function forward:touch()
motionX = 0
motionY = -speed
end
forward:addEventListener("touch",forward)
--Moving Right
function right:touch()
motionX = speed
motionY = 0
end
right:addEventListener("touch",right)
--Moving left
function left:touch()
motionX = -speed
motionY = 0
end
left:addEventListener("touch",left)
--Moves Frog each time frame is called
function movefrog (event)
frog.x = frog.x + motionX
frog.y = frog.y + motionY
end
Runtime:addEventListener("enterFrame", movefrog)
--Stops frog from moving continuously
local function stop (event)
if event.phase == "ended" then
motionX = 0
motionY = 0
end
end
Runtime:addEventListener("touch", stop)
--Making sure the frog does not go off the screen
local function stopfrog (event)
if frog.x <= 16 then
frog.x = 16
end
if frog.x >= 304 then
frog.x = 304
end
end
Runtime:addEventListener("enterFrame", stopfrog)
You need to set a condition under the waterCollide function in order to negate the respawning.
Depending on how complex you'll need it to be, you could simply check if frog's position is within log1's boundaries, or perhaps you could have log1 and any future logs have their own collision event which sets a flag on collision to not trigger the respawn then clear the flag when collision with any log ends.
Here's an example:
local onLog = 0
function frogDie()
lives = lives - 1
display.remove(frog)
frog = display.newImageRect("FrogTest.png",32,48)
frog.x = display.contentWidth*0.5
frog.y = 504
physics.addBody(frog, "dynamic")
frog.isFixedRotation = true
end
function waterCollide(event)
if onLog < 1 then frogDie() end
end
function logCollide(event)
if event.phase == 'began' then
onLog = onLog + 1
else
onLog = onLog - 1
end
end
log1:addEventListener("collision", logCollide)
--log2:addEventListener("collision", logCollide)
Using a number to track whether the frog is on a log should be safer than a boolean, since logs might end up overlapping and clearing the flag before it can be reset properly on multiple collision passes.