通过马尔可夫链模拟步骤

Question

我正在尝试通过马尔可夫链模拟一个步骤，目的是多次循环该过程，直到满足条件。 （即，找出平均需要多少步才能达到特定状态）。

在这种情况下，一个州只能走一条路。例如，状态 4 可以向前转换到状态 5，但不能向后转换到状态 3。这意味着转换矩阵的左下半部分设置为零。这也是为什么下面的方法将任意大的值置于“先前”状态的原因。我试图通过检查转换矩阵的指定行中哪个概率最接近随机数来找到正确的新状态。

get_new_state <- function(current_state, trans_matrix)
{
  # generate a random number between 0-1 to compare to the transition matrix probabilities
  rand <- runif(1)
  
  # transition to where the 
  # random number falls within the transition matrix
  row <- current_state # initial condition determines the row of the trans_matrix
  col = current_state # start in the column 
  # loop thru all columns and find the correct state
  potential_states <- rep(0, each=ncol(trans_matrix)) # holds the value of the potential state it transitions to

  # in this case, we can't transition to a previous state so we set the previous state values arbitrarily high 
  # so they don't get selected in the which.min() function later
  potential_states[1:col] <- 999
  
  for(k in row:ncol(trans_matrix)) # loop thru non-zero matrix values
  {
    if(trans_matrix[row,k] > rand)
    {
      potential_states[k] <- trans_matrix[row,k] / rand
      potential_states[k] <- 1 - potential_states[k]
    }
  }
  
  # new state is equal to the index of the lowest value
  # lowest value = closest to random number
  new_state = which.min(potential_states)
  return(as.numeric(new_state))
}

我不确定这种方法是否合理。我假设有一种更好的方法来模拟没有在potential_states[] 中放入任意大值的 kluge。

Answer 1

这样的事情会更好吗（它是单线马尔可夫转换）：

> new_state <- sample(1:number_states, size=1, 

             prob=transition_matrix[old_state,])

然后把它放在一个带有计数器的（例如）while() 循环中。