156 reputation
5
bio website slimjim.name
location
age 25
visits member for 2 years, 5 months
seen Jun 27 at 9:12

Feb
6
awarded  Scholar
Feb
6
accepted Company name already taken at github
Jan
2
awarded  Student
Jan
1
asked Company name already taken at github
Sep
9
answered Design: multiple algorithms on the same large data sets
Mar
17
comment I don't understand value iteration
Did my explanations clarify things?
Mar
17
comment I don't understand value iteration
Theta is just a value that tells the algorithm when the V converges. For each additional steps the algorithm (foreach higher start k) generates a better approximation of the V_(k=infinit) and if the difference from the last run is smaller than the threshold (theta).
Mar
17
comment I don't understand value iteration
Additionally for i in range(iteration): will have the same result as just setting i=iteration. It reminds of en.wikipedia.org/wiki/IDA* <--(the star is missing in the link for some reason)
Mar
17
comment I don't understand value iteration
looked through your code a couple of times but couldn't find anything major except for probability * reward + discount * val(i - 1, nextState) should be probability * (reward + discount * val(i - 1, nextState)), but it should still run
Mar
17
comment I don't understand value iteration
I don't see any problems with the pseudo code so perhaps you can include more? What happens if you increase theta? Whats the printout for max(Vk[s] for s in states) foreach iteration? (does it iterate over k atleast ones?) What is the stop condition for the recursion (V0)
Mar
17
comment I don't understand value iteration
Your could will lookdown one step further until it converges below theta
Mar
17
comment I don't understand value iteration
Vk(s) is the expected value (could be seen as a potential) of state s if you look down k steps with an optimal policy
Mar
17
comment I don't understand value iteration
You can extend the recursion depth in python docs.python.org/2/library/sys.html#sys.setrecursionlimit to see if it helps.
Feb
2
awarded  Supporter
Feb
2
awarded  Editor
Feb
2
awarded  Teacher