By declining the Ram Mandir invitation, the Congress may be diluting the traction it may still have.