To configure your card so that it only can be downloaded by end-users that have been invited you can set "By invitation only" under Card > Share.
If set, each user must be invited via email and card referrals will be disabled.
Invite Email Screenshot
The below is how the invite email looks in Gmail.
Related
Card Referrals
How to handle referrals to your cards.