While you were born in the Virginia colony, you have close family in England. Breaking with Britain would make that relationship difficult.
Your family owned a wheat and livestock farm west of the Appalachian Mountains and the Proclamation of 1763 forced you to move back to Virginia. You have since rebuilt your farm, but slowly. Boycotts on British goods have helped you grow your farm and become more profitable.
A family friend is a customs official and works for the British crown. When he was found to be a loyalist, a mob threatened him if he did not change his loyalties. He was forced to send his family away for safety, and he is believed to have gone into hiding.
Your father was killed in the French & Indian War and that has made you dislike war. You've heard your oldest son say the patriots are thinking of fighting the British, and you're worried your son will join them.
You do not use slave labor on your farms. However, you know that slave labor is a large part of Virginia's economic success. The impact of freedom for these people is something to consider.