Augustus had arranged in about 5 BCE or 6 BCE for Julia to marry Paullus, who had a family relation to her as her first half-cousin, as both have Scribonia as grandmother: Julia's mother was a daughter of Scribonia by Augustus; Paullus' mother, Cornelia Scipio, was a daughter of Scribonia resulting from her earlier marriage to Publius Cornelius Scipio Salvito.

Paullus and Julia had a daughter, Aemilia Lepida and (possibly) a son, Marcus Aemilius Lepidus (although the latter may also have been the son to Marcus Aemilius Lepidus (consul 6).

According to Suetonius, she built a large pretentious country house.

Augustus disliked large overdone houses and had it demolished.

In 8, according to ancient historians, Julia is exiled for having an affair with Decimus Junius Silanus, a Roman Senator.

She is sent to Trimerus, a small Italian island, where she gives birth to a child.

Augustus rejects the infant and orders it to be exposed, or left on a mountainside to die.

Silanus goes into voluntary exile (he will return under Tiberius' reign).

Sometime between 1 and 14, her husband Paullus is executed as a conspirator in a revolt.

Modern historians theorize that Julia's exile was not actually for adultery but for involvement in Paullus' revolt.

According to some, Livia Drusilla plotted against her stepdaughter's family and ruined them.

This leads to open compassion for the fallen family.