The solution described above by Beda works specifically when filtering by the post date, as the underlying query uses a date_query which can handle this, whereas with a date custom field the underlying query is a meta query, which can't.
I don't see how you can modify the query for this to work before the query is run.
The only option I see is for your View to return all posts (i.e. without the on-this-day filters), and for you to then iterate over the query results and remove all posts which are not from this day.
So you'll need code hooked into that filter which uses a foreach loop to iterate over all of the results, gets the custom date field (which is in a UNIX timestamp format), tests whether the date and month match today, and removes the post from the results if not.
You'll also need to update the found_posts and posts_count values, and re-index the results.
The test for whether to remove posts from the results is different, which you would need to update for your date tests, but the structure of the code is similar.