Well, believe it or not, this is actually good news. You can't get a perfect score without meeting up with the pirate. The message about him taking all your loot to his pirate chest isn't just a throwaway line. There really is a pirate chest in the maze after that point that you can find, along with all the rest of your stuff. And the chest itself is a treasure item.

To get from the entrance of the Pirate's Maze to the pirate's chest (and your loot), start at the west end of the Hall of Mists.

Then go d to enter the maze. You will see the dreaded "You are in a maze of twisty passageways, all alike."

Don't panic! Just type e, s, s, s, n, e, e, and nw. And there you are at the chest.

To escape, go se, n, and d. You will find yourself at the bird chamber room.