There may be two possibilities here for your rejection- you were nervous during the interviews Or you came across as an aggressive/manipulative/inflexible kind of person to the interviewers.
With your profile, you should probably wait for one year to give a shot at IIM A, B, C, L, before going for XLRI. Prepare for the interview with a calm and good presence of mind.
Even though CAT is unpredictable, take your chances, if you aspire to join Consulting or Finance firms. If you aspire for a Marketing/ Operations/ General Management role, then there would be hardly any difference.
Waiting for a year is a good idea. Along with the reviews and data, consider preference to be known as an IIM A alum than an XL alum. How well you manage your emotional and logical reasons for pursuing MBA is what you need to think about.