You’re driving up the Hudson Valley, maybe heading toward Beacon or Cold Spring, and suddenly you see it. A crumbling, jagged stone fortress sitting right in the middle of the river. It looks like something plucked out of a Scottish moor or a fever dream about medieval Europe.
That’s Pollepel Island.
Most people just call it Bannerman’s. Honestly, if you call it Pollepel to a local, they might take a second to realize what you’re talking about. It’s one of those places that feels like it shouldn’t exist in New York. We have skyscrapers and subway grates, not abandoned arsenals modeled after baronial castles. But there it is, sitting sixty miles north of New York City, slowly being reclaimed by vines and gravity.
The Weird Truth About the "Castle"
First off, let’s clear one thing up. It was never a residence. Francis Bannerman VI, the man who bought the island in 1900, wasn't a king or a knight. He was a businessman. A very successful, very specific kind of businessman. He dealt in military surplus.
Bannerman’s business was so massive that the city of New York basically told him he couldn't keep his inventory in Manhattan anymore. Why? Because he had millions of live cartridges and tons of black powder stored in a warehouse. It was a ticking time bomb. He needed a place away from people where things could, theoretically, blow up without leveling a city block.
He found Pollepel Island.
He spent years designing the buildings himself. He didn't use an architect. You can kind of tell if you look closely at the remaining walls; the proportions are slightly off, and the "decorative" flourishes are actually functional vents or supports. He wanted a giant advertisement. He literally had "Bannerman’s Island Arsenal" cast into the breakwater wall in massive letters so every passenger on the Hudson River Railroad would know exactly who owned the place.
💡 You might also like: How far is New Hampshire from Boston? The real answer depends on where you're actually going
Why Pollepel Island is Falling Down
If you visit today, you’ll notice the place is a shell. It’s a ruin. People often ask why the state or a billionaire doesn’t just "fix it."
Nature is winning.
In 1920, just two years after Francis Bannerman died, a massive explosion rocked the island. Somewhere around 200 pounds of powder and shells went off. It sent a chunk of the wall flying across the river onto the railroad tracks. That was the beginning of the end. Then, the ferry sank in a storm in 1950. Then, a massive fire gutted the structures in 1969.
The elements are brutal out there. The Hudson River isn't a lake; it’s a tidal estuary. The salt, the wind, and the freezing-thawing cycles of New York winters have turned the mortar into dust. In 2009, a huge portion of the residence's walls collapsed overnight. Today, the Bannerman Castle Trust works tirelessly to stabilize what’s left, but they aren't rebuilding. They’re "preserving the decay."
It’s a race against time. Every year, there is a little less of the castle to see.
Getting There Without Getting Arrested
You can't just kayak over there and start wandering around. Well, you could, but it's a terrible idea. The New York State Office of Parks, Recreation and Historic Preservation owns the island now, and they take the "no trespassing" rule seriously.
📖 Related: Hotels on beach Siesta Key: What Most People Get Wrong
Mainly because the buildings are death traps.
The walls are unstable. There are open pits. It’s dangerous. If you want to see it, you have to book a tour through the Bannerman Castle Trust. They run boats out of Beacon and Newburgh.
The Tour Experience
You get on a small boat (the Estuary Steward). The ride over is gorgeous, especially in the fall when the Highlands are turning orange and red. Once you land, you have to climb a lot of stairs. About 72, to be exact. If you have bad knees, this is your warning.
Once you’re up there, the views of Storm King Mountain are staggering. You’ll see the "residence," which was the family's summer cottage, and the massive "arsenal" warehouse. You can't go inside the buildings—again, the whole "falling on your head" risk—but you get close enough to see the intricate brickwork and the empty window sockets staring out at the water.
Legends, Ghosts, and "Polle Pell"
The island has a weird energy. Even before the castle, the Dutch had stories about it.
The name "Pollepel" supposedly comes from a girl named Pell. Legend says she was rescued from the ice on the river and landed on the island. Another version says the name refers to a "pot ladle" (pollepel in Dutch). Sailors used to think the island was haunted by "Heckst" or goblins who controlled the winds in the Hudson Highlands.
👉 See also: Hernando Florida on Map: The "Wait, Which One?" Problem Explained
If you believe the local lore, the ghosts of the munitions workers or the Dutch sailors still hang around. Honestly? The wind whistling through those empty stone arches is enough to make anyone a believer for a second. It's spooky.
What You Need to Know Before You Go
Don't just show up in Beacon expecting a ticket. These tours sell out weeks in advance, especially for the popular "movie nights" or "sunset tours" they do on the island.
- Footwear: Wear real shoes. Not flip-flops. The paths are gravel and uneven.
- Water: Bring your own. There aren't vending machines on a ruin.
- Camera: Bring a long lens if you have one. The best shots of the arsenal's "Bannerman" lettering are from the boat or the path near the residence.
- Weather: If it’s raining, the tour is likely cancelled. The island is exposed and there is very little shelter.
The Actionable Strategy for Visiting
If you want the best experience, aim for a kayak tour. Several outfitters in Cornwall-on-Hudson and Cold Spring lead guided paddles to the island. It’s a workout. The current in the Hudson is no joke, and if the tide is against you, you’ll feel it in your shoulders for a week. But approaching that massive stone wall from the waterline, at the same level as the old docks, is a completely different feeling than stepping off a motorboat.
Check the tide charts. If you’re paddling yourself (and staying in the designated areas), you want to time it so the tide helps you.
Also, look into the volunteer days. The Trust often looks for people to help with gardening or light maintenance. It’s one of the few ways to actually spend a significant amount of time on the island without a tour guide ushering you along.
Pollepel Island is a reminder that even the biggest empires—or at least the biggest surplus businesses—eventually crumble. It’s a beautiful, haunting mess. Go see it before another wall decides to join the riverbed.
Next Steps for Your Trip:
- Check the Schedule: Visit the official Bannerman Castle Trust website to see the 2026 tour dates. They usually run from May through October.
- Pick Your Starting Point: Beacon is easier for those taking the Metro-North train from Grand Central. Newburgh is better if you’re driving and want to grab dinner on the waterfront afterward.
- Book the "Living History" Tour: If they are offering it, choose the tour where actors play the Bannerman family. It sounds cheesy, but the historical context they provide about the Spanish-American War surplus makes the ruins make way more sense.
- Prepare for the Highlands: Since you're already there, plan to hike Mount Beacon or Storm King the same weekend to get the bird's-eye view of the island you just explored.