oh yeah they turn into gravel! It's an easy mistake.
My first year working at the chestnut orchard, I thought I would make some kind of shortbread cookie with chopped chestnuts on top for our harvest party. Nope! It was like chewing on sand with gravel nonpareils. Total fail!
My successes have been when I've made chestnut crème, and rolled that into things like chestnut Mont Blanc, chocolate chestnut pie, and this year, I made chestnut crème brûlée.
they're fantastic as a noodle substitute in minestrone. What I do is blanch and peel them, and then throw them into the soup around the same time that I throw in the green beans to cook.