Secret Evidence

I saw the phrase 'secret evidence' in one of the newspapers this week. I wondered how something secret could possibly be evidence. The question revolves around the value of testimony as evidence. If somebody I trust tells me that flamingos are pink birds, is that evidence? If somebody else, who overhears the conversation, says "Yes, flamingos are pink birds", is that more evidence? If somebody tells me "I have evidence that this is a flamingo, but it is secret", is that evidence for me?
As I see it, the first kind of testimony is evidence, but evidence of very limited value. It is evidence because it has the possibility of confirmation, because I can increase or decrease the strength of my belief in the assertion by adding further evidence, such as going to Lesbos and seeing if I can actually look at pink birds which people call flamingos. The second kind of testimony, simple agreement, seems to me to have no evidential value whatsoever; it brings nothing more to the argument. The third kind of testimony is more like the second than the first. It leaves no possibility of strengthening my belief by gaining further evidence, because the evidence is 'secret'.
I can see no reason why we should permit the state to act on 'secret' evidence; especially when that state could use such an epistemological nightmare to deprive us of our liberty.