Would you consider this copyright infringement?

Yes, you infringed their copyright by using the image from Family Guy. It is pretty cut and dried. You have no counter claim.