With the help of chr_sue, I find that in my reference model, before I write each trans into the analysis port which is finally connected to a fifo in the scoreboard, I forget to clone them. If I only write handles that points to the same object into a fifo, the content of the trans will be modified later. As they are already in the fifo, those changes will not be noticed, so the cause of the bug is very difficult to be found. The bug itself is even not obvious to be awared of.
Many thanks to chr_sue. He helps a lot.