As much as I hate it, the real way to achieve the level of compliance you describe is through surveillance, akin to what China does or what Black Mirror depicts.
Me neither, but it works. Japanese cities, even Tokyo, are SO much cleaner than cities in Europe, it's ridiculous. (They do have public trash cans in every convenience store, by the way.)
Culture is the reason. Japanese people have a more collectivist instead of individualist society, which promotes a shared sense of ownership in society. (At least, this is what they taught me in my Japanese courses in college)
Changing culture is hard. But, some cultural shifts have been brought about by public awareness campaigns, protests, etc.
Why not assess a fine to the original manufacturer of the litter (their logo is all over it), then use the proceeds to fund ongoing cleanup. Brands that are able to court responsible end users will find an edge in the market so will be compelled to use their marketing arms to raise awareness of the issue, perhaps even going so far as to fund their own cleanup efforts or incentivized packaging return.
The average person is reasonable and could probably be trusted to do the right thing. It's the 50% below average whom I'm worried about. You'll probably find a Pareto-like distribution on littering, i.e. ~80% of the litter is produced by ~20% of the people.
Another example that comes to mind is graffitis. My city is full of them. It takes seconds to make one, and hours to remove it. A few dozens of people could make a lot of damage within a few months.
I don't think this is the only way. I spent a couple of weeks in Rwanda last summer and was stunned by how clean the streets are both in Kigali and in the countryside.
I heard several years ago the picture was similar to neighboring countries (not good). This indicates that the cleanup is possible without introducing total surveillance.
I really don't think that's necessary. Just by having reasonable proximity to Public trashcans radically reduces littering.
Some level of perceived enforcement is required, but in reality it just needs to be a sign with a high fine on it. And finally some level of public funded cleanup groups and you can get pretty far with keeping spaces clean.