Sorry Dude, RR has an easement, not that they 'own' the land necessarily, but they have the property rites above ground, The easement is I think like a 100, 120 or 150 feet (i don't remember exact at the moment) wide for a single track, the track however doesn't necessarily run down the center of that width, there are agreements of course that allow for 'crossings' but even these are controlled points over specific roads and pathways, where you are expected to 'cross' and get on about your way. That's the 'Law'. That said!...this is 'Real Life'!. I grew up around, and lived next to, tracks my whole life and never followed there rules, as a kid and adult both, I blew them off just because the place I wanted to go was on the other side, other times following them was a more direct route then a Street or sidewalk, (Id hitch them sometimes, But Don't Try This At Home.. HA!). I would say that I had been stopped, and or yelled at, by both police and RR personnel several dozens of times over the last 35 years, and never once was I arrested. This is one of those things that is disregarded so often, by so many, they don't have the time or jail space to bother with it, You get a 'warning' every time. So if I was you, Id just do my thing, (don't push it though, get off the ballast when a train approaches, there are suicidal types out there, and if you make a train apply its breaks, then that would elevate the situation to a serious legal charge) If ever confronted, just say "Sorry Man.. I didn't Know?...". Just play stupid and leave the area. Now... That Said... Tracks are way cool to walk on, like a park setting, You could get some cool pictures too.
I say "go for it!"