You cannot modify the styles of the checkboxes, which are determined by the browser.
You can, however, use CSS to hide the actual checkbox and display a fake checkbox of your own design in its place with the same functionality.
There are many articles and examples online about the CSS required for such a solution.
However, they typically require that a particular HTML structure for the checkboxes, namely the input followed by the label.
The normal markup generated by Toolset for checkboxes is to have the input inside the label tags, which makes a CSS solution more difficult.
However, you can add the attribute output="legacy" to your checkboxes custom field and it will render the HTML in the required format of input then label.