The short answer if you are driving with a non transferable driver's license is YES. You do not say where you are from but suffice it to say that American state licenses and Canadian ones are not transferable. For more info about the rules here is the FINAL WORDon the ISL/ISP debate.
Also, before driving in Italy, make sure you read the section DRIVING IN ITALYand check out the road signs section within so you know some of the rules (they are different).
In June,2002 we were pulled over on the autostrada because we had a red number plate(registered in France and leased) and the police requested an international driver's licence.He spent quite some time checking this and the lease papers and we felt pleased that we had one. For the little cost and effort involved I would recommend it.
I can add my own experience to Penny's. Once I was exiting the toll booth at Orte on my way to Narni. A car sideswiped me trying to get ahead of me. RIGHT in FRONT of a policeman who was parked off to the right of the toll booth area "taking a break?". Anyway, I dutifully pulled over as the other car kept on going. The policeman showed absolutely NO interest in going after the offender. All he was interested in was checking me out. He asked for my papers. Briefly looked at my Missouri drivers license with distain and carefully studied my IDP. Then he handed them back to me and waved me on with a shrug. I was happy I had the IDP and even happier that I always rent from AutoEurope and get the full coverage.
We got a parking ticket (32 Euros) in Montalcino--Oops forgot to put the disk in the window and the first thing the parking attendant asked to see was our international drivers license. She knew French and Italian and the only word she spoke to us in English was "sorry" It was rather comical!