The OS X application spoofs your Mac to look like an Apple TV using Bonjour, so jailbreaking—or downloading any additional Cydia application—is not necessary (though, installing AirVideoEnabler is recommended if you want to stream from 3rd party apps).
What AirPlayer does is create and advertise a custom Bonjour AirPlay service that pretends to be an Apple TV. Bonjour is Apple’s zero configuration networking solution for allowing devices and applications to communicate with each other over local area networks. When Apple created AirPlay, it basically set up a new way for Apple TV to interact with iOS using Bonjour communications.
Sadun notes that since Bonjour has been ported to Windows and Linux, it is technically possible for this method to work with other operating systems, but another developer will have to implement it.
Remember, AirPlayer is currently in the alpha stages, so don’t expect it to work flawlessly.