SlimJim
Reputation
Top tag
Next privilege 200 Rep.
 Feb 6 awarded Scholar Feb 6 accepted Company name already taken at github Jan 2 awarded Student Jan 1 asked Company name already taken at github Sep 9 answered Design: multiple algorithms on the same large data sets Mar 17 comment I don't understand value iteration Did my explanations clarify things? Mar 17 comment I don't understand value iteration Theta is just a value that tells the algorithm when the V converges. For each additional steps the algorithm (foreach higher start k) generates a better approximation of the V_(k=infinit) and if the difference from the last run is smaller than the threshold (theta). Mar 17 comment I don't understand value iteration Additionally `for i in range(iteration):` will have the same result as just setting `i=iteration`. It reminds of en.wikipedia.org/wiki/IDA* <--(the star is missing in the link for some reason) Mar 17 comment I don't understand value iteration looked through your code a couple of times but couldn't find anything major except for probability * reward + discount * val(i - 1, nextState) should be probability * (reward + discount * val(i - 1, nextState)), but it should still run Mar 17 comment I don't understand value iteration I don't see any problems with the pseudo code so perhaps you can include more? What happens if you increase theta? Whats the printout for max(Vk[s] for s in states) foreach iteration? (does it iterate over k atleast ones?) What is the stop condition for the recursion (V0) Mar 17 comment I don't understand value iteration Your could will lookdown one step further until it converges below theta Mar 17 comment I don't understand value iteration Vk(s) is the expected value (could be seen as a potential) of state s if you look down k steps with an optimal policy Mar 17 comment I don't understand value iteration You can extend the recursion depth in python docs.python.org/2/library/sys.html#sys.setrecursionlimit to see if it helps. Feb 2 awarded Supporter Feb 2 awarded Editor Feb 2 awarded Teacher