My security software keeps on telling me there are unknown computers connected to my network. Should I be worried about this?

Is your network secure (I'm inclined to believe it's not)? If it's not secure, then what do you expect? People love free wi-fi. If it is secured, and you have unknown machines connected to it, then you have a lot to worry about.

You're right it's not, but I'm scared to knock them off in case they are really other computers in our network. Is this a valid worry or should I just do it?

Your router should be able to show you the IP and/or MAC addresses of any machine on the network. Take an inventory of those and compare them with the machines in your home. Any that don't match should be kicked off. After that, you should secure your network.*

*Unless you're my neighbor in the apartment building, in which case, I don't want to lose my connection.

They are showing. They match the computers in the network, except for one that shares the addresses of the router. Is this ok?

Sounds fine to me.

What makes you think its not the router itself? (or are you saying that there is a computer with the same IP as the router {which would probably mean its not connected to the network}).

Thank you for your help.

Why don't you secure your network?

I'm not sure how that would improve the situation.

I just mean in general, why don't you secure your network? It would prevent most people from accessing it.

I don't know how, want to tell me?

Sure. The first, most basic thing you should do is add a password. Routers usually come with a default username of "admin" and something generic for the password like "password," or nothing at all. Access your router and create a password. I don't know what kind you have, but on my Linksys, the setting is under Wireless > Wireless Security. There, you'll also select what kind of encryption you want to use (such as WEP or WPA). WPA or WPA2 are better choices. Your router should have documentation or help on the router access page with more info.

Under Wireless > Basic Wireless Settings you will find the option of disabling your SSID Broadcast. Your SSID is your network name. Anyone passing by within broadcast range is able to see your network name and try to access it, so if you disable broadcast it won't show up. Personally, I keep mine enabled because it's easier for other computers in the household to locate and connect.

Link.
Link.