The grammar structure をよそに is used to express that someone acts without regard to or in disregard of something – typically the feelings, concerns, expectations, or circumstances of others. It carries a nuance of criticism or disapproval, suggesting that the person is ignoring something they probably should not be ignoring.
The word よそ literally means "elsewhere" or "somewhere else," so をよそに conveys the image of pushing something aside, treating it as if it were someone else's business.
をよそに attaches to nouns (often nouns describing feelings, worries, or situations):
をよそに almost always implies that the person should have paid attention to what they are ignoring. It is used when describing someone acting selfishly, recklessly, or obliviously in the face of others' feelings or a serious situation. Because of this critical tone, it is rarely used to describe oneself in a positive light.