Hi mwddrwg! to BleepingComputer (BC) I think you'll find your fellow members to be really kind and will do their best to help with questions!
Let's see if either of these pictures will work for you (same picture, two different sizes)
Some details: The picture of the logo above appears to have already been cropped, thus it's not as clear as the official logo from the official website: https://www.clubwebsite.com/caernarfontown/News/view/939020 If you right click on the official logo, copy image address and paste it in a new tab (or window) you'll see a checker board pattern in the background which is "transparent" then notice there is the white border surrounding the actual logo (the border you would like to get rid of).
Here's the tricky part about that white border: I experimented in Photoshop attempting to remove it and it appears that the logo, including the border was created in Photoshop, which did not allow me to remove the white background as being transparent. You'll find that BC members support one another, thus there's a strong chance a member with greater experience in Photoshop (or other editing software) will come along and produce a miracle if the pictures above will not fulfil your need.
No doubt you have already experienced that although there are a lot of picture editing opportunities available, there sure isn't an easy fix to edit a "round" logo with a square border!!
Now then, I have never used Publisher, I do not know what you would need to embed a picture, or what size you are seeking.
Here are the direct links for the two pictures above if they will help you.
150x112 = https://s14.postimg.org/9ytp1q6wx/1474386537_caernarfon-town-f.c.png
320x240 = https://s9.postimg.org/yqbasasf3/1474386537_caernarfon-town-f.c.png
Please let us know your progress and reach out for anything else!