I agree. That *is* the problem of mimicry & the reason Turing regretted publishing the Turing test.
It seems that we can make statistical systems that imitate behaviors to arbitrary degrees.
It quacks like a duck, it walks like a duck. So computationalists believe it *is* a duck. But you can't eat the duck. And it does not shit like a duck either.
Usually, such arguments are dismissed as "metaphysical" (a four-letter word!), but I think good metaphysics are crucial.