One of the most visible differences between different churches is in how each practices baptism. Not only do some sprinkle while others immerse (dunk), churches even differ in whom they baptize. Some churches baptize only those who profess their faith in Jesus Christ (this is often called "believers baptism") while others baptize believers and their children. While what a person believes or even practices in regard to baptism is not essential to one’s salvation, it is still an important doctrine. On a practical level, parents must wrestle with whether or not they believe their own children should be baptized.
At UPC, while we believe that parents should have their children baptized, we also believe that parents should have the freedom to follow their own conscience in this matter. We believe that UPC should be a place where Christians can worship and serve together regardless of their convictions about baptism.
What follows is the reasoning why we believe Christian parents should present their children for baptism.
The definition of a sacrament
Baptism is one of the two sacraments that our Lord has given to the church. The other is the Lord’s Supper. Saint Augustine said that a sacrament was a "visible form of an invisible grace," or "a visible sign of a sacred thing."  John Calvin says that it is "an outward sign by which the Lord seals on our consciences the promises of his good will toward us in order to sustain the weakness of our faith; and we in turn attest our piety toward him in the presence of the Lord and of his angels and before men." 
It is important at the outset to state that the Reformed doctrine of baptism rejects all forms of sacerdotalism. Sacerdotalism is the doctrine that the sacraments in and of themselves infuse grace into the recipients. In other words, sacerdotalism claims that one is saved through the sacraments. The Reformers clearly rejected this, as do we, when we claim the biblical doctrine of justification by faith alone.
As a sacrament, baptism in and of itself does nothing. The very act of receiving baptism does not save a person. It is not a magical formula. The Westminster Confession of Faith states:
The grace revealed in or by sacraments in their right use does not come from any power in them. Neither does the effectiveness of a sacrament depend on the devoutness or the intention of whoever administers it. Rather the power and effectiveness of the sacraments are the result of the work of the Spirit and rest on God's word instituting them, since His word authorizes their use and promises benefits to worthy receivers of them. 
All of this to say that a sacrament, in and of itself, neither saves a person, nor guarantees their salvation (in the future).
Baptism in the Old Testament
Children in the Old Testament
As one reads through the Old Testament, one can see that children have always been counted among the people of God. Throughout the Bible, believers as well as their children enjoy God’s blessing and protection. For example, when God made his covenant with Noah, it was with Noah and his descendants (Genesis 9:9).
In Genesis 17:7-14, God established his covenant with Abraham and his descendants. The sign of the covenant was given not only to Abraham, but even to his children. After that point, baby boys were circumcised on the eighth day. This covenant that God established with Abraham, while having nationalistic aspects, was also a spiritual covenant (Galatians 3:16; Romans 4:16-18, 2 Corinthians 6:16-7:1).
This covenant did not end with the Old Testament, nor is it just for the Jews. In Romans 4:16, we discover that all Christians, whether Jews or Gentiles, are now included among God’s covenant people. Those who have the faith of Abraham are considered to be the true descendents of Abraham. That means that all Christians are descendants of Abraham. We are the true Israel of God.
The church, then, is not a separate people of God from the nation of Israel. Rather, the church is the name that the Bible gives to God’s people in the New Testament. In the Old Testament, the people of God were called Israel. In the New Testament, Israel has been expanded to included Gentiles as well as Jews. As Paul says in Romans 9 (quoting from Hosea 2:23), "As he says in Hosea: "I will call them ‘my people’ who are not my people; and I will call her ‘my loved one’ who is not my loved one," and, "It will happen that in the very place where it was said to them, ‘You are not my people,’ they will be called ‘sons of the living God.’"
In Romans 11, Paul writes that the Gentiles are engrafted into a pre-existing covenant that God had with Israel. While it is a new covenant, it is not completely new. It was not built from scratch, but is more akin to "New and Improved." It is far superior to the old. All this is to say that God’s covenant with His people in the New Testament is built upon His covenant with His people in the Old Testament.
In the Old Covenant, children were included. They received the sign of the covenant (circumcision) as well as the promises. How could a Jewish parent whose child was included in the old covenant see the new covenant as superior if it now excluded his children? God did not scrap the old covenant to build the new. He built the new upon the old.
The meaning of circumcision
The Old Testament sign of the covenant was circumcision. By sign, we mean that those who had the mark of circumcision were included in the covenant community and counted among the people of God. Those who did not receive the sign were excluded from the community and cut off. Circumcision was established as the sign of the covenant in Genesis 17:11. Deuteronomy 10:16 (see also Deuteronomy 30:6 and Jeremiah 4:4) states that circumcision was symbolic of cleansing, and the cutting away of sin. While it was an outward act, it was primarily to serve as a symbol of an inward reality, namely God's working on the heart. It is a symbol of putting to death the flesh.
No one would dispute that children were given the sign of the covenant in the Old Testament. The old covenant clearly included children. However, that did not mean that the children would definitely be saved or that they had faith. All infants who were circumcised were not saved just as all the descendants of Abraham were not saved. Salvation, even in the Old Testament, came through faith (Genesis 15:6). So infants had the sign of salvation, the sign of the covenant, even though they had not yet come to faith in God. Even though faith was necessary for salvation, faith was not necessary for being a member of the covenant community or for having the sign of the covenant.
Baptism and Circumcision
Some may ask the question, if circumcision was the sign of the old covenant, why was it discontinued in the New Testament. In Hebrews 10, we see that there no longer needs to be a sacrifice for sin. Christ has shed His blood so there is no longer any point to shedding blood. Since circumcision was a bloody rite, it is made obsolete by the shedding of Christ's blood. However, that does not mean that there is no longer a sign of the covenant.
From the New Testament, one can see that baptism has essentially the same meaning as circumcision. Romans 4:11 says that circumcision was a seal of faith. That is what baptism is. Both circumcision and baptism symbolize the inner cleansing from sin. Acts 22:16 shows that baptism, like circumcision, is symbolic of cleansing.
In Colossians 2:11-13, baptism and circumcision are used interchangeably. Both are used to denote the death of the sinful nature and the new life in Christ. This passage more than any other shows that in their spiritual significance, baptism and circumcision are identical.
Both baptism and circumcision were initiation rites into the church. This has been seen regarding circumcision in the passages already discussed and regarding baptism in its practice in the New Testament as well as passages such as Galatians 3:27-29.
Baptism and circumcision both are tokens of the covenant (Genesis 17:11; Acts. 2:38-41; Galatians 3:27-29).
Baptism and circumcision are both seals of the righteousness of faith (Romans 4:11).
Both are symbols of purification (Colossians 2:11).
Baptism in the New Testament
Arguments from silence
The major reason for controversy surrounding the baptism of infants is the silence of the New Testament. If the Bible clearly stated that they should be baptized or should not be baptized then there would be no argument. However, it does not.
Some look at the Bible and claim that, since we do have accounts of adult converts being baptized then adults must be the only proper recipients of baptism. However, because children are not specifically mentioned in any record of baptism, it does not necessarily follow that they are excluded from the sacrament.
Those who claim that children should be baptized look at the silence of the New Testament in an entirely differently light. They would claim that the inclusion of children of believers "is so much in line with the thought and practice of the Old Testament that it is taken for granted in the New, as the household baptisms of Acts suggest even if they do not prove. In this regard the unity of the Old and New Testaments has an importance which should not be overlooked, though in the first instance it may mean simply that the apostles who first administered Christian baptism were steeped in the theological teaching of the first revelation of God to Israel."  In other words, the apostles are silent on the matter because there has been no change in the position of children in the New Covenant from their position in the Old.
Both of these are arguments from silence. That does not mean that they are false or that they are equally compelling. It only means that neither argument forces by necessity a particular conclusion. The question is, can a case be made for the inclusion of children on the grounds of a legitimate inference even though the NT is silent?
An example of inclusion by legitimate inference is the admission of women to the Lord's Table on equal terms with men. It is universally accepted that women can partake of communion just like men. However, there are no explicit texts that include women. No women were present when Christ instituted the Lord's Supper. He did not command any women to "do this." In the few instances recorded of the celebration of the Lord's Supper, there is no explicit reference to women being present and participating. While this example proves nothing in regard to the baptism of infants, it does demonstrate that even when the Bible is silent, legitimate inferences may be drawn.
Jesus and children
Because there is no explicit statement regarding infant baptism in the New Testament, that does not mean that there is no evidence for it. Jesus’ love for children is explicit in the NT (Matthew 18:3-6; 19:13-15; Mark 10:13-16; Luke 18:15-17). Interestingly, Jesus says that the Kingdom of God belongs to such children, a strong indication that they are included in God's covenant. While not conclusive in and of itself, these verses do give weight to the supposition that children are part of the covenant and therefore should have the sign of the covenant.
In the book of Acts, an interesting phenomenon occurs. When the head of a household comes to faith in Christ, not only is he or she baptized, but so is the entire household. There is the case of the Philippian Jailer (Act 16:33-34), and the case of Lydia (Acts 16:14-15) and Stephanas (1 Corinthians 1:16). It is possible in both cases that only the adults were baptized and that all who were baptized came to faith in Christ before being baptized, but the text never says so. In view of the Jewish doctrine of the covenant and the inclusion of the family, it seems most plausible that if the children were now excluded, that would have to be stated. Interestingly, Peter, in his first gospel message, demonstrates the continuity of the covenant and the inclusion of children when he speaks of the promise being not only for his adult hearers, but for their children as well (Acts 2:39).
Children included in the promise
Children have always been included in the covenant. This is seen in the administration of the old covenant (Genesis 9:9; 17:7; Exodus 20:4; Deuteronomy 29:10-13). This did not change in the new covenant. Peter states that the promises of the covenant were not only to those who heard his sermon and understood it, but also for their children (Acts 2:38-39)
Just as is the case today, many adults come to Christ after being married. Often times, the spouse does not come to Christ at the same time. In Corinth, there were some women who had become Christians whose husbands had not. They were concerned about the state of their children. In 1 Corinthians 7:14, Paul indicates that even the spouse enjoys some sort of blessing from being married to a Christian and that the children are "holy." Obviously, Paul is not saying that the spouse or the children are holy in their character and behavior just because they are married to Christians, but that they are set apart, they are legitimately a part of the covenant community. For those who do not include children, this passage is impossible.
Admonitions to children
In the Epistles, Paul and Peter both admonish children to follow Christ in obedience. These admonitions are given to them as part of the church. He seems to regard them as part of the church and expects them to live up to their covenant obligations (Ephesians 6:1; Colossians 3:20).
Evidence from History
Arguments from history, like arguments from silence, do not force any necessary conclusions. The Bible, not the practice of saints in the past, is the only rule for faith and practice. However, historical arguments are valuable as corroborative testimony.
An interesting point is that Jews during the time of Christ practiced baptism as a rite for initiation of converts from other nations. In this rite, children were included. 
Irenaeus and Origen, who lived in the late 100's speak of the baptism of infants. Irenaeus, who was "a hearer of Polycarp and Polycarp of the apostle John," also testifies of infant baptism.  That means that baptism of infants was definitely practiced in the time not too distant from the closing of the New Testament. Again, this evidence does not necessarily prove anything, but it is compelling.
 Calvin's Institutes, 4.14.1
 Calvin's Institutes, 4.14.1
 The Westminster Confession of Faith, 27.3
 Children of Promise, Geoffrey W. Bromiley p. 2
 Children of Promise, Geoffrey W. Bromiley p. 4
 Children of Promise, Geoffrey W. Bromiley p. 4