I hesitate to mention mpv a second time in the same thread, but using mpv should make this fairly easy. You can control mpv over a unix socket, which should allow you to implement nearly any custom control scheme you can think of.

https://mpv.io/manual/master/#json-ipc