I'm very sensitive to sugar and starch when "at rest". If I wake up first thing in the morning and drink an orange juice or have a bowl of white rice before bed, my blood sugar and mood are out of control. But the same foods mid-day before/after/during intense mental or physical work are very well tolerated if not beneficial. So my priors agree with yours, timing and context absolutely matters.