Did you know that most museums in Paris are free on the first Sunday of the month? Remember this when planning your next budget cultural weekend. Plus there’s no queuing at ticket desks, which means more time to track down off-piste treats.

I began my pursuit of the unexpected at the Musée de la Chasse et de la Nature (Museum of Hunting and Nature). Now, a place with this name wouldn’t normally get my guns firing. But “freedom Sunday” encouraged me to try it out.

It’s an old-established collection in a 17th-century residence; I’d envisaged wood-panelled walls, antique pictures and stuffed animals beneath glass domes. I wasn’t wrong (although it’s had a fresh and funky makeover), but what hit me as I stepped inside was its joyful family atmosphere. As I ventured into the shadowy Room of the Wild Boar a grandfather was eagerly asking a child “Are you afraid?” Both were clearly relishing the bristly creature facing them down.

A dazzling installation: an eagle and a dove become one another

I continued to a side room with a ceiling covered in owl heads and wings. Then I discovered a stag that had been turned into bagpipes. An adjacent video showed how the carcass could be inflated to play a wailing lament when pressed. I watched as a mother had to drag her two transfixed offspring away and on to the Room of the Unicorn (or was it the Wolf?).

A stairwell was illuminated by flickering candelabra. Stuffed panthers were poised to pounce out of the gloom. A delight in the natural world was balanced by glimpses of a darker side, both its threatening recesses and how humans menace it in return.

A machine made of feathers for meditating on migratory birds

Part of the eccentricity derived from a temporary exhibition, Art Orienté Objet, of strange animal artworks inserted throughout the museum’s permanent display. Inspired by the weird world of Hieronymus Bosch’s painting The Garden of Earthly Delights, they focused on unexpected aspects of the relationship between animals, nature and man.

If you visit Paris before the exhibition closes do drop by. It’s just behind the Pompidou Centre, among quirky boutiques, cafes and townhouses with notable residents (I saw the name Renzo Piano on a door). But the museum’s collection of art and artefacts is fascinating in itself. And no doubt there’ll soon be another offbeat exhibition to plumb the more macabre realms of our natural world.

Art Orienté Objet by Marion Laval-Jeantet & Benoît Mangin, until 2 March.  Musée de la Chasse et de la Nature, 62 rue des Archives – 75003 Paris.

Photographs by Nicolas Hoffmann