I'm now 7 weeks post prostatectomy after a finding of grade (ISUP) 3 cancer (was Gleason 4+3).  Looks like the margins and lymph were clear, so that's positive - just awaiting first follow-up PSA in 6 weeks or so.

As noted above in some comments, it's important for you and your medical professionals consider a number of factors, of which PSA is one.  My father died of (not with) prostate cancer which metastasised to his liver so I have been doing PSA for a number of years.  My latest one last year prompted the doc to review my history which showed a steady doubling every 3.8 years over the last few years.  Note that is wasn't wildly over the limit (of 4.0 in my situation), but the increase and family history meant that he was concerned enough that I was off the the urologist.  Scan (MRI) late last year showed likely cancer and follow-up biopsy confirmed.

With regards to the DRE, your doctor should let you know that it really only checks for abnormalities posteriorly.  It is worth noting that mine (two lesions, one bigger than the other - the main one came out at 2.3cm in the histopathology report post removal) were both in the anterior part of the prostate (apparently not quite so common).  I only mention this for information purposes from my personal experience.

All the above is simply to reinforce what others have said - keep checking and talking to your doctor about the results ... and remember that there is not necessarily a definitive answer as to whether you should be being sent to a urologist or not - just keep asking the questions of your medical professionals to try and make sure.