U is a 3×3 matrix, and R, G, B are just the vectors (1,0,0), (0,1,0) and (0,1,1). So:
UR = the 1st column of U
UG = the 2nd column of U
UB = the 3rd column of U
The vector in our 27-dimensional space for (UR)(UG)(UB) will contain a component in the RRR direction of U_{11} U_{12} U_{13}, but all six terms will only differ by a sign, and the sum will be zero. Similarly for GGG and BBB.
What about RGG, or RRG, or any other direction with two identical colours? In that case, there will always be two contributions that are the same except that they have opposite signs. So again, the sum will be zero.
The only directions where there is no cancellation will be those with three different colours. For the RGB direction, the 6 contributions will be:
+ U_{11} U_{22} U_{33}
- U_{11} U_{23} U_{32}
+ U_{12} U_{23} U_{31}
- U_{12} U_{21} U_{33}
+ U_{13} U_{21} U_{32}
- U_{13} U_{22} U_{31}
But this is just the determinant of U, which is equal to 1!
For the RBG direction, we just get minus the determinant of U, and for all six terms we get the determinant or its opposite.
So any matrix U in SU(3) leaves w unchanged, and this 3-particle state does not feel the strong force.