Yes, sorry, I hope this helps:
On my page users are able to share their work and some info about themselves.
To showcase their previous work they should be able to upload their portfolio.
The portfolio in this case is provided as either a pdf file or, if that does not exist as a link to an external website.
I do not want to clutter the frontend form with unnecessary fields.
The user would go into an editing form for their profile and should only be able to see a single field labeled as "portfolio".
In this field I want the user to upload their pdf file or provide a link.
A solution to this is that you can create a repeatable field group for the user profile page to add their portfolio items. In this repeatable field group you will have the fields that would make up the portfolio items.
Then you can create a form for this field group only and then add it to the user's portfolio page. From there they can use the form to add a new portfolio item.