I have a 460 with I mild cam and I've had issues with it idling and it's hard to get started in the morning. It has Holley 650 double pumper  and a family friend of mine that use to build race motors said I need a bigger carb what are your guys thoughts ?

bigger carb isnt gonna help with idling and hard cold starts.  if anything it'll make it worse.  thats the whole point of a choke is to restrict airflow to help with cold idles and starts.  you just gonna hold yer foot on the pedal for a little while for it to get some temp into the motor before it'll idle.  thats how it goes with a big block/cam carb motor. either that or raise the idle rpm.

I have a 460 with I mild cam and I've had issues with it idling and it's hard to get started in the morning. It has Holley 650 double pumper  and a family friend of mine that use to build race motors said I need a bigger carb what are your guys thoughts ?

I agree that a 460 will perform better with a bigger carb, but the concerns you have have nothing to do with a small carburetor.  You can start with double checking your tune (condition of spark plugs, timing, idle speed, dirty fuel filter...).  I'm running two 660's and my boat starts and idles without issue (most times  )