John Brown of Haddington, Systematic Theology, Book VI, Chap. 1, p. 475:
The Christian Sabbath begins in the morning after midnight. 1. Christ rose early in the morning, Matt 28:1; Mark 16:2,9. 2. It begins where the Jewish sabbath ended, which was when it began to dawn towards the first day of the week, Matt 28:1,3. 3. The evening which follows the day of our sabbath pertained to it, John 20:19.
Lewis Bayly, The Practice of Piety, pp. 163-164:
The Jews kept the last day of the week, beginning their Sabbath with the night (Gen 2:2; Lev 23:32; Neh 13:19), when God rested; but Christians honour the Lord better, on the first day of the week (Matt 28:1), beginning the Sabbath with the day when the Lord arose (Acts 20:7,11) They kept their Sabbath in remembrance of the world's creation; but Christians celebrate it in memorial of the world's redemption; yea, the Lord's day being the first of the creation and redemption, puts us in mind, both of the making of the old, and redeeming of the new world.
John Willison, A Treatise Concerning the Sanctification of the Lord's Day, pp. 78-79:
But all these things being absurd, I do upon solid ground, assert, that the whole natural day, consisting of twenty-four hours, is to be set apart for the Sabbath day; and that we ought to measure this day, and begin and end it, as we do other days, that is, from midnight to midnight; during which time we are abstain from our own works, and sanctify the Lord's Sabbath: For the fourth command binds us to consecrate the seventh part of every week to the Lord, who challengeth a special property in one of seven, and asserts his just title thereto, saying, "The seventh day is the Lord's:" And also Isa. lvii.13 he expressly calls it, "My holy day." It is all holy; and therefore no part must be profaned or applied to common uses. It is all the Lord': and so it is unlawful for us to rob him of any part of it, and alienate it to our private use.
John Willison, An Example of Plain Catechising Upon the Assembly's Shorter Catechism, p. 188:
Q. How much of the day appointed for the sabbath is to be kept holy to the Lord?
A. One whole day in seven; a whole natural day, consisting of twenty-four hours, commencing from midnight to midnight, ought to be dedicated unto the Lord, seeing he claims a seventh part of our time. It is true, time for eating and sleeping must be allowed upon the sabbath as well as on other days, being works of necessity, seeing without these we cannot perform the duties of the sabbath.
Thirdly, The day to be kept holy is one whole day. Not a few hours while the public worship lasts, but a whole day. There is an artificial day betwixt sun-rising and sun-setting, John xi.9; and a natural day of twenty-four hours, Gen. i. which is the day here meant. This day we begin in the morning immediately after midnight; and so does the sabbath begin, and not in the evening, as is clear, if ye consider,
1. John xx.19. The same day at evening, being the first day of the week: where ye see that the evening following, not going before this first day of the week, is called the evening of the first day.
2. Our sabbath begins where the Jewish sabbath ended; but the Jewish sabbath did not end towards the evening, but towards the morning, Matth. xxviii.1. In the end of the sabbath, as it began to dawn towards the first day of the week, &c.
3. Our sabbath is held in memory of Christ's resurrection, and it is certain that Christ rose early in the morning of the first day of the week.
Let us therefore take the utmost care to give God the whole day, spending it in the manner he has appointed, and not look on all the time besides what is spent in public worship, as our own; which is too much the case in these degenerate times wherein we live.
Alexander McLeod, The Ecclesiastical Catechism; Being a Series of Questions, Relative to the Christian Church, Stated and Answered, With the Scripture Proofs:
140. At what period of the twenty-four hours does the Lord’s day or sabbath commence?
Our Lord arose from the dead on the morning of the first day of the week [a]: it is more conducive to solemnity to observe one whole day, than parts of two labouring days [b]: the fourth commandment requires not a part of two days, but one whole day [c]; and the evening after Christ’s resurrection, upon which he appeared in the midst of his worshipping disciples, is called, in scripture, the evening of the same day [d]: the christian sabbath comprehends twenty four hours, from midnight to midnight.
[a] John 20. 1. "The first day of the week cometh Mary Magdalene early, when it was yet dark—and seeth the stone taken away from the sepulchre." [b] Deut. 5. 14. "The seventh day is the sabbath of the Lord—in it thou shalt not do any work." [c] Exod. 20. 8. "Remember the sabbath day to keep it holy." [d] John 20. 19. "Then the same day at evening, being the first day of the week—came Jesus and stood in the midst, and saith unto them, Peace be unto you."
John Wallis, A Defense of the Christian Sabbath, p. 28:
I have insisted the longer on this, because I find him afterward moving another question about what time the Sabbath is to begin and end, and lays great stress upon it, as we shall see anon.
Of which I think we need not be further solicitous than to begin and end this day, according as other days are accounted to begin and end in the places where we live....
...I take it to be very plain from what I have said, that at the time of Christs Death and Resurrection, it [the Christian Sabbath] was accounted to begin very early in the morning, while it was dark, and continue till very late at night, according as we now account our days, from Mid-night to Mid-night.
Obs. 220. -- The Fourth Commandment requireth us to sanctify one whole day in seven, which God hath expressly appointed to be a holy Sabbath to himself.
By one whole day, as the stated time of worshipping God, we are to understand the same that we are to understand by any other whole day -- namely, a period consisting of twenty-four hours, or what is commonly called a natural day. And this day we should begin and end at the same time that we begin and end any other day, -- namely, at midnight.
William Plumer, The Law of God, as Contained in the Ten Commandments, Explained and Enforced, pp.309-310:
When does the Sabbath begin?
There is some diversity in the Christian world respecting the time, at which the Sabbath begins. Some date it from sunset on Saturday till sunset on Sabbath. When asked for their authority, they refer to a phrase which occurs several times in the first chapter of Genesis: "And the evening and the morning were the first day." This has not been considered sufficient proof by the great mass of the Christian world. Nor ought it to be, as all the world knows that no day of creation began in the evening; but all of them began in the morning. That saying of Moses therefore only declares that the day was made up of two parts, the after part, and the fore part. Indeed the evidence in the New Testament seems to be clearly against this view. "Our Sabbath begins where the Jewish Sabbath ended; but the Jewish Sabbath did not end towards the evening, but towards the morning. Matt. 28:1. ‘In the end of the Sabbath, as it began to dawn towards the first day of the week,’ etc. In the New Testament, the evening following, and not going before this first day of the week, is called the evening of the first day, John 20:19. ‘The same day, at evening, being the first day of the week,’ etc. Our Sabbath is held in memory of Christ’s resurrection, and it is certain that Christ rose early in the morning of the first day of the week."
Brian Schwertley, The Christian Sabbath: Examined, Proved, Applied:
5. The time of the Lord’s day
The sabbath law teaches that man is to sanctify to the Lord one whole day in seven. A question that needs to be answered is: “When does the Christian sabbath begin?” Some argue that the Christian sabbath begins on Saturday evening, while others argue that it runs from midnight Saturday to midnight Sunday. Those who argue that it runs from evening to evening point to the Jewish ceremonial sabbaths for support: “On the fourteenth day of the first month at twilight is the Lord’s Passover” (Lev. 23:5). The Hebrew word translated “twilight” (NKJV, NASB, NIV, NTHSMT ) or “evening” (RSV) literally means “between the evenings.” “The meaning of the phrase is much discussed. Most commentators think it means ‘in the evening’ (cf. Deut. 16:6, ‘at sunset’), or more precisely, the period between sunset and complete darkness. The orthodox Jewish view is that it means ‘between midday and sunset,’ and this is supported...on the grounds that it would have been impossible to kill all the passover lambs in the temple between sunset and darkness. In NT times the passover sacrifice began about 3 p.m.”  The evidence for an old covenant evening-to-evening sabbath is quite strong (cf. Lev. 23:32; Ex. 12:6, 30:8). Hendriksen believes that the Jewish sabbath began at 6 o’clock Friday evening: “According to the ancient Hebrew way of speaking, there were ‘two evenings’ (cf. Exod. 12:6 in the original). The first ‘evening’ which we would call ‘afternoon’ began at 3 p.m., the second at 6 p.m. Something of this is probably reflected in the phrase ‘When evening fell,’ for we cannot imagine that Joseph of Arimathea, a Jew, would have approached Pilate on Friday, 6 p.m., asking for the body of Jesus when the sabbath was beginning.” 
Although the Jewish sabbath was probably from evening to evening (or sunset to sunset), the passages in the New Testament which discuss the Lord’s day (the new covenant sabbath) point to a midnight-to-midnight observance. A passage which indicates that the inspired apostles no longer held to the old covenant system of a sunset-to-sunset sabbath is John 20:19: “Then, the same day at evening, being the first day of the week, when the doors were shut where the disciples were assembled, for fear of the Jews, Jesus came and stood in the midst, and said to them, ‘Peace be with you’” John is very specific in noting that this event took place on the first day of the week. “‘On that day’ would be enough, yet John adds, ‘the first one of the week.’”  “It was evening. In light of Luke 24:29, 33, 36 we have a right to conclude that it was no longer early in the evening when the great event recorded in the present paragraph took place. As the Jews compute the days, it was no longer the first day of the week. But John, though a Jew, is writing much later than Matthew and Mark, and does not seem to concern himself with Jewish time-reckoning.”  It is very significant that John emphasizes that the disciples gathered on the first day of the week, yet also records that it was evening, for if the apostolic church had maintained a sunset-to-sunset sabbath, then John would not have regarded it as the first day, but as the second. There then would be no reason at all for John to emphasize the time, for while the New Testament often emphasizes and singles out the first day (Mt. 28:1; Mk. 16:2; Lk. 24:1; Jn. 20:1, 19, 26; Ac. 20:7; 1 Cor. 16:2; Rev. 1:10), the second day holds no significance at all.
Another passage which indicates that the apostolic church had forsaken the sunset-to-sunset sabbath for a midnight-to-midnight  sabbath is Acts 20:7: “Now on the first day of the week, when the disciples came together to break bread, Paul, ready to depart the next day, spoke to them and continued his message until midnight.” Luke describes a church service that occurred on the first day of the week, yet says very specifically that Paul did not finish his message until midnight. If the Christian church had followed the Jewish synagogue practice, Paul would have concluded his message before sunset on Sunday, and not late at night.  “Certainly, one would almost expect the midnight-to-midnight demarcation, not only in the light of the particulars surrounding Resurrection Sunday, but especially considering that Troas was a Roman colony possessing the Jus Italicum and which therefore certainly followed the Roman midnight demarcation as a colony. It is clear that the congregation at Troas met for worship at night well after sunset, for ‘there were many lights in the upper chamber, where they were gathered together’ (Acts 20:8). Seeing that ‘the disciples came together to break bread’ in ‘the upper chamber,’ and seeing that there is no instance whatsoever in Scripture of religious meetings on Saturday night after sunset, it is reasonably certain that the disciples at Troas gathered on Sunday nights perhaps even before and certainly after sunset, even as their risen Lord had appeared to His Emmaus disciples on Resurrection Sunday and broken bread with them in the late afternoon, and long after the sunset of ‘the same day at evening, (still) being the first day of the week,’ had congregated with the Jerusalem disciples in the upper room.” 
Furthermore, it is recorded that Paul departed at daybreak, or the break of the next day. If Luke had been following the sunset-to-sunset day demarcation system of the Jews, Paul would be described as leaving “later on the same first day of the week.”  But Luke says of Paul on Sunday evening that he was “ready to depart the next day” (i.e., early Monday morning at daybreak). Thus there is considerable evidence that the inspired apostles abandoned the Jewish method of day demarcation for a midnight-to-midnight system. Although the matter of sabbath day demarcation may seem trivial, it is important that the church and society follow the inspired apostles’ example for the sake of uniformity, determining when church discipline is appropriate, preparing properly for the Sabbath, and refuting heretics (such as Seventh-day Adventists). “The Jews are supposed to begin every day, and consequently their sabbaths, at the evening, in remembrance of the creation, Gen. i. 5, as Christians generally begin their days and sabbaths with the morning, in memory of Christ’s resurrection.” 
Francis Nigel Lee, The Covenantal Sabbath: The Weekly Sabbath Scripturally and Historically Considered, pp. 273, 327:
Counting the ‘three days’ as Friday, Saturday, and Sunday, it is clear that Christ must have been in ‘the heart of the earth’ on part of Friday as well as on part of Sunday. But as He was not buried until Friday evening (Mt. 27:57-60), it seems clear that the end of each of the three days does not run from evening to evening (as Jews and Seventh-day Adventists allege), but from a point between evening and dawn—probably midnight—to the corresponding point twenty-four hours later. The ‘three days and three nights in the heart of the earth’ of Matt. 12:40 etc. would then be: sunset on Friday to midnight after Friday = the first ‘day and night;’ the midnight after Friday to the midnight after Saturday = the second ‘day and night;’ the midnight after Saturday to the sunrise on Sunday = the third ‘day and night;’ and the whole period in the grave from sunset on Friday to before dawn on Sunday morning = ‘three days and three nights,’ which expression is an idiom denoting a period of exactly three days and three nights (seventy-two hours) OR denoting any consecutive shorter parts thereof, such as the approximately thirty to thirty-six hours during which Christ was in the tomb.
The answer to the fourth sub-query: "When was Sunday first observed, and how?", must be that Christ and His Spirit progressively taught its observance in the hearts of God’s children by example rather than by precept particularly from Resurrection Sunday onwards. And in so teaching, They then probably indicated that, like the Adamic sabbath before the fall and like the Resurrection Sunday of Christ the Second Adam, the New Testament Sunday observance was to run from midnight to midnight (Matt. 28:1,6,13; Mark 16:1-2,9; Luke 24:1, 13, 24-36; John 20:1, 19; Acts 20:6-7, 11 cf. Ex. 11:4; 12:6, 12-16, 27-29, 42 cf. Gen. 1:3-5, 31; 2:1-3);...