I would like to see either as you describe (which I think would be the most relaxing), or have the direction-changing buttons also count as a click each time they are pressed. I found it tempting to switch direction, turn a piece, then switch back, often using two clicks instead of the three it would take to turn it clockwise-only, but very inefficient since I switched back and forth many times. Making those count as clicks, though, does up the difficulty and would make hitting the target require you to use them only once, so perhaps removing them altogether is the solution.
If you did it as clockwise by default instead of listing the ideal turn count you could list the minimum number of “counter” turns. Then have a button to do the next turn as a counter turn.
I would have suggested having 2 zones per tile: one to rotate left, one to rotate right. But the tap zone is quite small already, so it's prone to errors.
One slower but more UX-friendly option would be to allow long-press on a tile, it would zoom in, and you can pick left or right with each button.
Double tap is also an option but you need to remove the ability to select the content of the page.
Not sure how easy it is to implement, but here is another idea:
hold on the tile few seconds, then you have the choice to swipe left or right (has also some edge cases, when the elements are in the corner)
Tap-and-hold a tile, move finger outside of it and then rotate by moving finger around tile's center (like using a handle that you just extended from the tile).
