Spawn Function in Corona SDK - function

Okay so I kind of am almost there but need a little push:
My dustbins spawn but there are a few to begin with like I want and then after about 30 seconds of the numbers of dustbins increasing there are 1000's all clumped together.. What can i do to alter my function so that its only ever a few at a time:
function spawnDustbin()
dustbin = {}
for i = 1,800 do
dustbin[i] = display.newImage("dustbin.png")
dustbin[i].xScale = 0.55
dustbin[i].yScale = 0.55
dustbin[i].y = 555
dustbin[i].x = (math.random(1000,1500) * i)
dustbin[i].speed = 4
physics.addBody( dustbin[i], "static", { friction=1.0, density=1.0, bounce=0, radius=30,} )
dustbin[i].enterFrame = moveDustbin
Runtime:addEventListener("enterFrame", dustbin[i])
end
end
and the movedustbin function simply moves the dustbin in the negative x direction:
function moveDustbin(self,event)
if self.x < -2560 then
self.x = 2560
else
self.x = self.x - val
end
end
First 20ish seconds:
http://i.stack.imgur.com/7iEeP.png
After 20 seconds:
http://i.stack.imgur.com/aae0D.png
Thank you very much
James

That "spawnDustbin" function above is spawning 800 dustbins every frame. That will really crush your performance, and I am pretty certain you don't want to create that many objects every frame.
You might want to do something like this:
local dustbin = {}
local i = 0
local function spawnDustbin()
i = i+1
dustbin[i] = display.newImage("dustbin.png")
dustbin[i].xScale = 0.55
dustbin[i].yScale = 0.55
dustbin[i].y = 555
dustbin[i].x = (math.random(1000,1500) * i)
dustbin[i].speed = 4
physics.addBody( dustbin[i], "static", { friction=1.0, density=1.0, bounce=0, radius=30,} )
dustbin[i].enterFrame = moveDustbin
Runtime:addEventListener("enterFrame", dustbin[i])
end
timer.performWithDelay(500, spawnDustbin, 10)
YMMV but that will spawn 10 dustbins in 5 seconds.

Related

Issues with Q-learning and neural networks

I'm just starting out learning Q-learning, and I've been okay with using the tabular method to get some decent results. One game I found quite fun to use Q-learning was with Blackjack, which seemed like a perfect MDP type problem.
I've been wanting to extend this to using a neural network as a function approximator, but I'm not having any luck at all. The approach is to calculate the expected value for every action in a given state and then pick the best one with a small chance of picking something random (epsilon greedy). Nothing converges, it learns silly Q-values, and it can't even figure out how to play when the only card in the deck is 5.
I am genuinely stuck, after spending hours on this and tuning hyper parameters and everything else I can think of. I feel like I must have made a fundamental error with Q-learning that I can't see. My code is below:
import gym
from gym import spaces
from gym.utils import seeding
import numpy as np
import random
import pandas as pd
import sklearn
import math
import itertools
import tensorflow as tf
from matplotlib import pyplot as plt
############################ START BLACKJACK CLASS ############################
class Blackjack(gym.Env):
"""Simple Blackjack environment"""
def __init__(self, natural=False):
self.action_space = spaces.Discrete(2)
self._seed()
# Start the first game
self.prevState = self.reset()
def _seed(self, seed=None):
self.np_random, seed = seeding.np_random(seed)
return seed
# Returns a tuple of the form (str, int) where str is "H" or "S" depending on if its a
# Soft or Hard hand and int is the sum total of the cards in hand
# Example output: ("H", 15)
def getTotal(cards):
running_total = 0
softs = 0
for c in cards:
running_total += c
if c == 11:
softs += 1
if running_total > 21 and softs > 0:
softs -= 1
running_total -= 10
return "H" if softs == 0 else "S", running_total
def drawCard():
# Draw a random card from the deck with replacement. 11 is ACE
# I've set it to always draw a 5. In theory this should be very easy to learn and
# The only possible states, and their correct Q values should be:
# Q[10_5, stand] = -1 Q[10_5, hit] = 0
# Q[15_5, stand] = -1 Q[15_5, hit] = 0
# Q[20_5, stand] = 0 Q[20_5, hit] = -1
# The network can't even learn this!
return 5
return random.choice([5,6])
return random.choice([2,3,4,5,6,7,8,9,10,10,10,10,11])
def isBlackjack(cards):
return sum(cards) == 21 and len(cards) == 2
def getState(self):
# Defines the state of the current game
pstate, ptotal = Blackjack.getTotal(self.player)
dstate, dtotal = Blackjack.getTotal(self.dealer)
return "{}_{}".format("BJ" if Blackjack.isBlackjack(self.player) else pstate+str(ptotal), dtotal)
def reset(self):
# Resets the game - Dealer is dealt 1 card, player is dealt 2 cards
# The player and dealer are represented by an array of numbers, which are the cards they were
# dealt in order
self.soft = "H"
self.dealer = [Blackjack.drawCard()]
self.player = [Blackjack.drawCard() for _ in range(2)]
pstate, ptotal = Blackjack.getTotal(self.player)
dstate, dtotal = Blackjack.getTotal(self.dealer)
# Returns the current state of the game
return self.getState()
def step(self, action):
assert self.action_space.contains(action)
# Action should be 0 or 1.
# If standing, the dealer will draw all cards until they are >= 17. This will end the episode
# If hitting, a new card will be added to the player, if over 21, reward is -1 and episode ends
# Stand
if action == 0:
pstate, ptotal = Blackjack.getTotal(self.player)
dstate, dtotal = Blackjack.getTotal(self.dealer)
while dtotal < 17:
self.dealer.append(Blackjack.drawCard())
dstate, dtotal = Blackjack.getTotal(self.dealer)
# if player won with blackjack
if Blackjack.isBlackjack(self.player) and not Blackjack.isBlackjack(self.dealer):
rw = 1.5
# if dealer bust or if the player has a higher number than dealer
elif dtotal > 21 or (dtotal <= 21 and ptotal > dtotal and ptotal <= 21):
rw = 1
# if theres a draw
elif dtotal == ptotal:
rw = 0
# player loses in all other situations
else:
rw = -1
state = self.getState()
# Returns (current_state, reward, boolean_true_if_episode_ended, empty_dict)
return state, rw, True, {}
# Hit
else:
# Player draws another card
self.player.append(Blackjack.drawCard())
# Calc new total for player
pstate, ptotal = Blackjack.getTotal(self.player)
state = self.getState()
# Player went bust and episode is over
if ptotal > 21:
return state, -1, True, {}
# Player is still in the game, but no observed reward yet
else:
return state, 0, False, {}
############################ END BLACKJACK CLASS ############################
# Converts a player or dealers hand into an array of 10 cards
# that keep track of how many of each card are held. The card is identified
# through its index:
# Index: 0 1 2 3 4 5 6 7 9 10
# Card: 2 3 4 5 6 7 8 9 T A
def cardsToX(cards):
ans = [0] * 12
for c in cards:
ans[c] += 1
ans = ans[2:12]
return ans
# Easy way to convert Q values into weighted decision probabilities via softmax.
# This is useful if we probablistically choose actions based on their values rather
# than always choosing the max.
# eg Q[s,0] = -1
# Q[s,1] = -2
# softmax([-1,-2]) = [0.731, 0.269] --> 73% chance of standing, 27% chance of hitting
def softmax(x):
"""Compute softmax values for each sets of scores in x."""
e_x = np.exp(x - np.max(x))
return e_x / e_x.sum()
plt.ion()
# Define number of Neurons per layer
K = 20 # Layer 1
L = 10 # Layer 2
M = 5 # Layer 2
N_IN = 20 # 10 unique cards for player, and 10 for dealer = 20 total inputs
N_OUT = 2
SDEV = 0.000001
# Input / Output place holders
X = tf.placeholder(tf.float32, [None, N_IN])
X = tf.reshape(X, [-1, N_IN])
# This will be the observed reward + decay_factor * max(Q[s+1, 0], Q[s+1, 1]).
# This should be an estimate of the 'correct' Q-value with the ony caveat being that
# the Q-value of the next state is a biased estimate of the true value.
Q_TARGET = tf.placeholder(tf.float32, [None, N_OUT])
# LAYER 1
W1 = tf.Variable(tf.random_normal([N_IN, K], stddev = SDEV))
B1 = tf.Variable(tf.random_normal([K], stddev = SDEV))
# LAYER 2
W2 = tf.Variable(tf.random_normal([K, L], stddev = SDEV))
B2 = tf.Variable(tf.random_normal([L], stddev = SDEV))
# LAYER 3
W3 = tf.Variable(tf.random_normal([L, M], stddev = SDEV))
B3 = tf.Variable(tf.random_normal([M], stddev = SDEV))
# LAYER 4
W4 = tf.Variable(tf.random_normal([M, N_OUT], stddev = SDEV))
B4 = tf.Variable(tf.random_normal([N_OUT], stddev = SDEV))
H1 = tf.nn.relu(tf.matmul(X, W1) + B1)
H2 = tf.nn.relu(tf.matmul(H1, W2) + B2)
H3 = tf.nn.relu(tf.matmul(H2, W3) + B3)
# The predicted Q value, as determined by our network (function approximator)
# outputs expected reward for standing and hitting in the form [stand, hit] given the
# current game state
Q_PREDICT = (tf.matmul(H3, W4) + B4)
# Is this correct? The Q_TARGET should be a combination of the real reward and the discounted
# future rewards of the future state as predicted by the network. Q_TARGET - Q_PREDICT should be
# the error in prediction, which we want to minimise. Does this loss function work to help the network
# converge to the true Q values with sufficient training?
loss_func = tf.reduce_sum(tf.square(Q_TARGET - Q_PREDICT))
# This are some placeholder values to enable manually set decayed learning rates. For now, use
# the same learning rate all the time.
LR_START = 0.001
#LR_END = 0.000002
#LR_DECAY = 0.999
# Optimizer
LEARNING_RATE = tf.Variable(LR_START, trainable=False)
optimizer = tf.train.GradientDescentOptimizer(LEARNING_RATE)#(LEARNING_RATE)
train_step = optimizer.minimize(loss_func)
init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)
# Initialise the game environment
game = Blackjack()
# Number of episodes (games) to play
num_eps = 10000000
# probability of picking a random action. This decays over time
epsilon = 0.1
# discount factor. For blackjack, future rewards are equally important as immediate rewards.
discount = 1.0
all_rewards = [] # Holds all observed rewards. The rolling mean of rewards should improve as the network learns
all_Qs = [] # Holds all predicted Q values. Useful as a sanity check once the network is trained
all_losses = [] # Holds all the (Q_TARGET - Q_PREDICTED) values. The rolling mean of this should decrease
hands = [] # Holds a summary of all hands played. (game_state, Q[stand], Q[hit], action_taken)
# boolean switch to use the highest action value instead of a stochastic decision via softmax on Q-values
use_argmax = True
# Begin generating episodes
for ep in range(num_eps):
game.reset()
# Keep looping until the episode is not over
while True:
# x is the array of 20 numbers. The player cards, and the dealer cards.
x = cardsToX(game.player) + cardsToX(game.dealer)
# Q1 refers to the predicted Q-values before any action was taken
Q1 = sess.run(Q_PREDICT, feed_dict = {X : np.reshape( np.array(x), (-1, N_IN) )})
all_Qs.append(Q1)
if use_argmax:
# action is selected to be the one with the highest Q-value
act = np.argmax(Q1)
else:
# action is a weighted selection based on predicted Q_values
act = np.random.choice(range(N_OUT), p = softmax(Q1)[0])
if random.random() < epsilon:
# action is selected randomly
act = random.randint(0, N_OUT-1)
# Get game state before action is taken
game_state = game.getState()
# Take action! Observe new state, reward, and if the game is over
game_state_new, reward, done, _ = game.step(act)
hands.append( (game_state, Q1[0][0], Q1[0][1], act, reward) )
# Store the new state vector to feed into our network.
# x2 corresponds to the x vector observed in state s+1
x2 = cardsToX(game.player) + cardsToX(game.dealer)
# Q2 refers to the predicted Q-values in the new s+1 state. This is used for the 'SARSA' update.
Q2 = sess.run(Q_PREDICT,feed_dict = {X : np.reshape( np.array(x2), (-1, N_IN) )})
# Store the maximum Q-value in this new state. This should be the expected reward from this new state
maxQ2 = np.max(Q2)
# targetQ is the same as our predicted one initially. The index of the action we took will be
# updated to be [observed reward] + [discount_factor] * max(Q[s+1])
targetQ = np.copy(Q1)
# If the game is done, then there is no future state
if done:
targetQ[0,act] = reward
all_rewards.append(reward)
else:
targetQ[0,act] = reward + discount * maxQ2
# Perform one gradient descent update, filling the placeholder value for Q_TARGET with targetQ.
# The returned loss is the difference between the predicted Q-values and the targetQ we just calculated
loss, _, _ = sess.run([loss_func, Q_PREDICT, train_step],
feed_dict = {X : np.reshape( np.array(x), (-1, N_IN) ),
Q_TARGET : targetQ}
)
all_losses.append(loss)
# Every 1000 episodes, show how the q-values moved after the gradient descent update
if ep % 1000 == 0 and ep > 0:
Q_NEW = sess.run(Q_PREDICT, feed_dict = {X : np.reshape( np.array(x), (-1, N_IN) ),
Q_TARGET : targetQ})
#print(game_state, targetQ[0], Q1[0], (Q_NEW-Q1)[0], loss, ep, epsilon, act)
rolling_window = 1000
rolling_mean = np.mean( all_rewards[-rolling_window:] )
rolling_loss = np.mean( all_losses[-rolling_window:] )
print("Rolling mean reward: {:<10.4f}, Rolling loss: {:<10.4f}".format(rolling_mean, rolling_loss))
if done:
# Reduce chance of random action as we train the model.
epsilon = 2/((ep/500) + 10)
epsilon = max(0.02, epsilon)
# rolling mean of rewards should increase over time!
if ep % 1000 == 0 and ep > 0:
pass# Show the rolling mean of all losses. This should decrease over time!
#plt.plot(pd.rolling_mean(pd.Series(all_losses), 5000))
#plt.pause(0.02)
#plt.show()
break
print(cardsToX(game.player))
print(game.dealer)
Any ideas? I'm stuck :(

Calling a function multiple times with return value

from graphics import *
def draw():
returnStuff = {'again' : 0, '1st' : 1 }
draw.again = False
win = GraphWin("Quadrilateral Maker", 600, 600)
win.setBackground("yellow")
text = Text(Point(150, 15), 'Click 4 points to create a Quadrilateral')
text.draw(win)
#gets the 4 points
p1 = win.getMouse()
p1.draw(win)
p2 = win.getMouse()
p2.draw(win)
p3 = win.getMouse()
p3.draw(win)
p4 = win.getMouse()
p4.draw(win)
vertices = [p1, p2, p3, p4]
#draws the shape
quad = Polygon(vertices)
quad.setFill('red')
quad.setOutline('black')
quad.setWidth(3)
quad.draw(win)
text.setText('Click in the appropriate box.')
#Quit box
quitBox = Rectangle(Point(30, 500), Point(100,550))
quitBox.setFill('green')
quitBox.draw(win)
quitorNah = Text(Point(60, 490), 'Quit')
quitorNah.draw(win)
#again box
quitBox = Rectangle(Point(480, 500), Point(550,550))
quitBox.setFill('green')
quitBox.draw(win)
quitorNah = Text(Point(510, 490), 'Draw Again')
quitorNah.draw(win)
click = win.getMouse()
x = click.getX()
y = click.getY()
while True:
if 30 < x < 100 and 500 < y < 550:
returnStuff['again'] = 0
win.close()
break
elif 480 < x < 550 and 500 < y < 550:
returnStuff['again'] = 1
win.close()
break
return returnStuff
count = 1
returnValue = draw()
if returnValue['1st'] == 1:
count = 0
while count == 1 or returnValue['again'] == 1:
return_value = draw()
So I have this simple interactive program using Zelle graphics, it asks the user to click on 4 points in a window and from that it creates a shape. Then, the user is shown 2 boxes, one to quit and one to draw again. My draw again isn't working, and it has something to do with the return value. I am returning a dictionary, as I need access to 2 of the variables within the function. In the 'returnStuff' dictionary, I have a part called 'again', which is initially set to 0. If the user clicks in the run again box, it changes this value to 1, and then outside the function I have an if statement that should call the function again if that again value is 1. It does this properly the FIRST time, but the 2nd time around my program just stops all together, and I don't understand why.
Can anybody explain why this is happening?
I think you need a while...
while count==1 or returnValue['again'] == 1:
returnValue = draw()

What does this recursive function mean?

I've been tasked with implementing a recursive function in MIPS. This function is
function1(n) = n-5 (if n <= 3)
otherwise = 4*function1(n-1) - n*function1(n-3)
One of the test cases is if n = 6, then the result is 200.
How do you get 200 from entering 6 in this function? To me it looks like the answer should be 2. Is there something I am not understanding about recursion, or am I understanding the function wrong? I am so confused
You seem to be misunderstanding the function somewhere. Here are the steps I followed to arrive at 200:
function1(6) = 4*function1(5) - 6*function1(3) (by rule 2)
function1(5) = 4*function1(4) - 5*function1(2) (by rule 2)
function1(4) = 4*function1(3) - 4*function1(1) (by rule 2)
function1(3) = 3-5 = -2 (by rule 1)
function1(2) = 2-5 = -3 (by rule 1)
function1(1) = 1-5 = -4 (by rule 1)
Substituting back...
function1(4) = 4*-2 - 4*-4 = -8 - -16 = 8
function1(5) = 4*8 - 5*-3 = 32 - -15 = 47
function1(6) = 4*47 - 6*-2 = 188 - -12 = 200

Need help stopping functions in corona

I'm trying to stop 3 animations in my function when it reaches a certain point and then have a message display "Animations Stopped".
How would I do this? I know about display.NewText() but how would I go about stopping the animations and getting the message to pop up at the same time?
Here is the function I'm trying to stop.
WIDTH = display.contentWidth
HEIGHT = display.contentHeight
--displays background
local s = display.newImageRect("space.png" ,1136, 640)
s.x = 900/2
s.y = 500/2
--display main ship
local r = display.newImageRect("ship.png", 70, 70)
r.x = 20
r.y = 450
local minions = {}
function createMinions()
local x = 40
local y = 120
for n = 1, 20 do -- displays 20 minions
local minion = display.newImageRect("minion.png", 50, 50)
minion.x = x
minion.y = y
minions[n] = minion
x = x + 60 -- next enemy will be to the right
if x >= WIDTH then -- start a new row if necessary
y = y + 60 -- seperation between minions
x = 40
end
end
end
--display mothership
m = display.newImageRect("mothership.png", 150, 150)
m.x = 160
m.y = 10
function nextFrame()
-- begins movements of main ship to right
r.x = r.x + 5
if r.x > 350 then
r.x = -100
end
-- begins movement of minions to the left
for i = 1, 20 do
local minion = minions[i]
minion.x = minion.x - 8
if minion.x < -100 then
minion.x = 400
end
end
--begins movement of mothership towards small ship
m.y = m.y + 10
if m.y > 460 then
m.y = -100
end
--stops all animations
if m.y > 450 then
--r.x = r.x + 0
--m.y = m.y + 0
--minion.x = minion.x + 0
local s = true
--displays game over text
s = display.newText("Game Over", WIDTH/2, 400, native, 30)
end
end
createMinions()
Runtime:addEventListener( "enterFrame", nextFrame )
--hides status bar
display.setStatusBar( display.HiddenStatusBar )
Just to make it very short -
This is the code specific to your problem.
if(m.y < 450) then YOURCODE else DISPLAYTEXTCODE end
For future I suggest people to look at:
Corona Docs: API > Libraries > Transition > To - To move display objects.
and
Corona Docs: APi > Types > EventListener > addEventListener - To trigger code based on events.

MySQL, how would my function to evalue a number look?

I need a function in MySQL which will evaluate a status code...
In my program this is how I evaluate it. However, in SQL I'd want to select all the status code which had a specific Reason. You can see below the reasons and that more than one can be set.
Sub InterpretReasonCode(ByVal pintCode As Integer)
If pintCode >= 16 Then
pintCode -= 16
mbooBlacklistedDomain = True
End If
If pintCode >= 8 Then
pintCode -= 8
mbooSneakedURLChanged = True
End If
If pintCode >= 4 Then
pintCode -= 4
mbooRetriedFailedToAccess = True
End If
If pintCode >= 2 Then
pintCode -= 2
mbooRequestedByAuthor = True
End If
If pintCode >= 1 Then
pintCode -= 1
mbooBlackListed = True
End If
End Sub
My SQL statement would look something like this
Select * from MyTable where Eval_Func(StatusCode,8) = true;
In the future I will be expanding the function to include different flags, .e.g 32, 64, 128, 256 etc.
You don't need a function.
You can mask it directly using bitwise operators
For example:
-- Get all BlacklistedDomain
select *
from tab
where reason_bit_mask & 16;
And I advice against this on anything other than really small tables, because you can't index it properly.