Potter has emerged as a leading contender to take over at the Amex Stadium following the sacking of Chris Hughton.
Should compensation be agreed between the clubs, it will be up to Potter to decide whether he leaves Championship side Swansea.
Albion have been tracking Potters progress. The 43-year-old has reinvigorated cash-strapped Swansea since joining them last summer after steering Swedish side Ostersunds to three promotions and the Europa League.
Potter, 43, signed a three-year deal when he left Swedish side Ostersunds FK for Swansea in June 2018.
Graham Potter on the shortlist to become Albions next manager
Swansea have not commented, but are aware they will have a fight on their hands to keep Potter if he is offered the chance to manage in the Premier League after just 11 months in charge at the Liberty Stadium.
Potter, who spent seven years as manager of Swedish club Ostersund before moving to Wales, impressed during his first season at Swansea, helping the club rebuild following relegation from the Premier League.
Following the departure of several senior players, Potter handed first-team chances to a number of youngsters and his new-look team improved as 2018-19 progressed.
The Swans were in the hunt for a play-off place until the final week of the campaign, with Potter re-establishing the possession game which had been Swanseas trademark until the latter stages of their spell in the Premier League.
Graham Potter, the Swansea City manager, will be interviewed for the job at Brighton & Hove Albion within the next 24 hours after the club shocked Chris Hughton by sacking him yesterday morning.
Hughton, 60, was dismissed despite leading Brighton to Premier League safety and an FA Cup semi-final this season. He had been at the club since 2014 and had no idea until a 15-minute meeting with Tony Bloom, the chairman, at 9am. He had been planning for next season.
Hughton was one of only two black managers, along with Nuno Espírito Santo, working in English footballs top division. Swansea last night gave Potter permission to hold talks with Brighton, who may also consider other candidates for the role.
Get The International Pack for free for your first 30 days for unlimited Smartphone and Tablet access.