This article aims at addressing the age-old question: What happens after you type a URL into a web browser? The use case entails the background details of typing “google” in the address bar. Before we begin, it is ideal to note that ignoring the website’s downtime, the average load time of any webpage is less than 2.9 seconds, leaving each of the processes highlighted below occurring in nanoseconds or less. This article covers an in-depth analysis of behind-the-scene activities from the point a user presses the “g” button up until a result displays on the screen.
User presses letter “g”
The moment the first letter in google (letter G) is pushed, the browser receives the input entry event, and an auto-complete function immediately kicks in. Depending on your browser’s algorithm, various suggestions are presented to the end-user in the dropdown below the URL bar, varying from recently entered, web browser type, recently trending to most entered inputs. In this case, the auto-complete function completes the query to google.com.
The “enter” key bottoms up
Taking the point at which the “Enter” key on the keyboard hits the bottom of its range (called bounce) and debounce, an electrical circuit specific to the enter key is directly or capacitively closed. This closed-circuit allows the flow of minute current into the keyboard’s logic circuit, which then scans the state of each key switch by debouncing the unique electrical noise generated from the switch’s rapid intermittent closure. The electrical noise is converted into a keycode integer (in this case, 13), which is encoded by the keyboard controller for transport between the keyboard and the computer. The transmission channel, generally over a Universal Serial Bus (USB), can also be a Bluetooth connection, or legacy connectivity like PS/2 or ADB
The moment the processor finds a closed circuit, it compares the location of the most recently closed circuit on the key matrix to the character map in its read-only memory (ROM). A character map, being a comparison chart or lookup table for each key encoding (UTF, ASCII…), signals the processor the position of each key in the matrix and what each keystroke or combination of keystrokes represents. The keyboard, exclusively reserved for interrupt request line (IRQ 1) sends signals on its IRQ, which maps to an interrupt vector by an interrupt controller. The computer then maps out the interrupt vectors to functions uses the Interrupt Descriptor Table (IDT) provisioned by the kernel. As the interrupt arrives, the kernel inputs when the CPU indexes the IDT with the interrupt vector and runs the appropriate handler.
CPU message relay
The keyboard’s switch contacts are wired to a microprocessor arranged in an X-Y matrix, constantly beaconing to notice the updated values of the currently opened or closed keys. When it senses the enter key down, it sends a serial code to the PC (single byte for most keys). When it senses a key released that was previously down, the same code with an additional “0xF0” byte is received. The host PC captures these incoming codes and uses them to maintain its map of which keys are up and down. The PC then writes the ASCII code for each new ‘down’ key into a volatile memory buffer where any running application can retrieve it.
For example; ‘A’ – make code 0x1C; break code oxF0,0x1C
Browser input Logic
The browser, being equipped with the ability to decipher outputs based on predefined inputs, can process the logic to interpret “https” as HyperText Transfer Protocol (Secure), “/” as slash notation, “@” as email predecessor. In the case where no protocol or valid domain name is specified, the browser then feeds the text entered in the address bar to the browser’s default web search engine (Google, Baidu, Bing). The URL is usually appended with unique texts to notify the search engine of its origination, which in this case, is from a specific browser’s URL bar for a browser-specific rendering during output.
Check non-ASCII Unicode Characters
In the world of Multilingual Web Addressing, where non-ASCII characters are now added to Web addresses, browsers are now able to detect and convert non-ASCII Unicode characters ( a-z, A-Z, 0-9, -,) in hostnames. Since the hostname is google.com here, the Punycode is resolved by the domain name server into a numeric IP address. For encoding, If the string representing the domain name contains non-Unicode characters, the string is converted to Unicode by the user agent (UA). Normalization functions are then performed on the string to remove mismatch. This can include converting uppercase characters to lowercase, reducing alternative representations, and eliminating prohibited characters (whitespaces). Next, UA converts each of the labels (pieces of text between dots) in the Unicode string to a Punycode representation. To differentiate original ASCII labels from non-original labels, a unique marker (‘xn--‘) is prefixed for each label containing non-ASCII characters.
The browser then performs a lookup on its internal list of websites that have requested to be contacted via HTTPS only called “preloaded HSTS (HTTP Strict Transport Security)” list. If the list contains the website (google), the browser sends its request via HTTPS instead of HTTP, automatically appending the “s” at the suffix of the HTTP after the certificate authority (CA) is initialized and verified. Alternatively, the initial request is sent via HTTP. However, a website can use the HSTS policy without being in the browser’s HSTS list. This is established when the user’s first HTTP request to the website receive a response that the user only send HTTPS requests, this then forces the browser to establish communications only in HTTPS going forward.
The browser then checks if the domain exists in its local cache. If the domain exists, the most recent record is first pushed into the memory before comparing the variation with the live record. If the domain is not found in the local cache, the browser calls an OS-specific library function to invoke a lookup. This library function (gethostbyname in widows) initially checks if the hostname is resolvable by referencing the local host file then attempts to resolve the hostname via DNS. In a situation where the library function is unable to find a cached version or the hosts file, a request is made to the default DNS server, which is usually the local router or the ISP‘s caching DNS server in other instances.
The target IP address is required by the network stack library to initiate an ARP (Address Resolution Protocol) broadcast, while the MAC address of the interface is required to send out the initializing ARP broadcast. The ARP cache is then looked up for an ARP entry for the target IP address. If an entry is found, a “Target IP = MAC” is returned. If the ARP entry is not in the cache, the routing table is looked up to check if the target IP is on any subnets on the local routing table. If a matching entry is found, the interface associated with that subnet is used by the library, else, the interface containing the subnet of the default gateway is utilized. The unique 48 bit MAC address of the NIC (Network Interface Card) is looked up. The network library sends an L2 (datalink) ARP request in the format:
Sender MAC (source): MM:MM:MM:SS:SS:SS
Sender IP address (source): #.#.#.#
Target MAC (dest): FF:FF:FF:FF:FF:FF (Broadcast)
Target IP address (dest): #.#.#.#
Computer Network Connectivity
Considering how devices can join the network via wired or wireless connectivity, the process of establishing the ARP request varies depending on what technology exists between the computer and the router, being the path to the internet. If the computer is DC (Directly connected) to the router, the router instantly responds with an ARP reply. If a hub (deprecated) is serially between the computer and the router, the hub sends broadcasts containing the ARP request out all other ports irrespective of their current status, establishing connectivity for the router based on serial topology. If the computer is DC to a switch, the switch then checks its local CAM/MAC table in an attempt to match a corresponding port with the MAC address in question. If no entry is found, the ARP request is rebroadcasted to all other ports.
Opening of a socket
Since the network library now has the IP address of either the DNS server or the default gateway, the initial DNS process is resumed. If the response size exceeds the required size, TCP port 53 is opened to send a request to the DNS server, else, UDP is used. The browser then collates the IP address and port number of the destination server from the URL, makes a call to the system library function using the default port numbers depending on the protocols used (the HTTP: 80, HTTPS: 443), and requests a TCP socket stream named AF_INET/AF_INET6 (for IPv4 and IPv6 respectively)and SOCK_STREAM. This request first hits L4 (Transport Layer), where a TCP segment is created. The source port is chosen from within the kernel’s dynamic port range while the destination port is added to the header.
OSI Model downstream
The segment is sent to L3 (Network Layer) of the OSI model, which adds an IP header to the first segment. The destination server and the current machine IP address and is then converged to make up a packet. Next, the packet arrives at layer 2 (datalink Layer), alongside an additional frame header, which includes the MAC address of the machine’s NIC and the MAC address of the default gateway (local router). As previously addressed, if the know the MAC address of the gateway is unknown to the kernel, the kernel must broadcast another ARP query to find it.
At this stage, the packet is acknowledged and ready to be transmitted using most traditional media, including wifi(wireless), Ethernet cable(wired), or Cellular data network (wireless mobile devices).
Most SOHO (Small Office Home Office)or personal Internet connections involve packet movement from a computer, through a local network, and a modem (MOdulator/DEModulator), which converts binaries to an analog signal suitable for transmission via telephone (deprecated), cable, or wireless connections. On the ISP’s end of this connection is a similar modem reverses the analog signals back into digital data to be reprocessed by the next network node where the “from” and “to” addresses are further examined. If fiber optic or direct Ethernet connectivity is used, the data remains unchanged and is passed directly to the next network node for further examination.Dataflow
When the packet reaches the router managing the local subnet, it continues to travel to a collection of one or more IP prefixes run by one or more network operators that maintain a single, clearly-defined routing policy known as Autonomous Systems (AS). Each router on this path then extracts the destination address from the IP header and routes the traffic to the next appropriate hop. The TTL ( Time To Live) field embedded in the IP header continues to reduce by 1 for each router that successfully passes through. In the case of network congestion, where the current router has no space in its queue, or the TTL field reaches zero, the packet is dropped.
Following the TCP connection flow, the 3-way handshake is initialized. This involves the process of a client choosing an initial sequence number (ISN) and sending the packet to the server with the synchronize (SYN) sets. The server then receives the SYN request by choosing its ISN, Notifies SYN of this update, then copying the client ISN +1 to its ACK field with the addition of an ACK flag acknowledging the receipt of the initial packet. The client then acknowledges the connection by sending a packet containing an increased sequence and receiver acknowledgment number. On the source, SEQ is increased by the same number of data bytes. The recipient then sends an ACK packet, which is equal to the last received sequence from the source packet.
To close the current connection, the closing terminal sends a FIN packet while the recipient ACKs the FIN with its FIN, the closing terminal then ACKs the recipient’s FIN with its own ACK
CA (Certificate Authority)
The client computer sends a “ClientHello” plus its TLS version number to the server, the server then replies with a “ServerHello” message to the client with its TLS version, cipher mode, compression methods and the public certificate of the server, signed by a verifiable CA (Certificate Authority). In this certificate is a public key which the client uses to encrypt other unencrypted portion of the handshake before a mutual symmetric key is selected for use. The client then runs a verification of the server’s digital certificates against its list of trusted CAs. If trust is established, the client continues to generate a string of pseudo-random data encrypted with the server’s public key, which is used to determine the symmetric key type.
The server then decrypts the random bytes using its private key. Next, it uses this data to generate a localized copy of the symmetric master key. At this point, the client sends an “end-of-message” notification to the server and encrypts the transmission with a symmetric key. The server accepts this message, generates its localized hash, decrypts the incoming hash from the client, and verifies a match with both hashes. If the hashes match, the server sends a finished message encrypted with the symmetric key to the client. If the hashes do not match, a certificate validity error is logged. The TLS session is then established for the transmission of encrypted application data using a chosen symmetric key algorithm (3DES, RC4, IDEA, CAST5…)
In the case where the web browser used was written by Google (Google Chrome), a negotiation request is sent to the server to upgrade the regular HTTP protocol to SPDY (pronounced speedy), as opposed to sending a usual HTTP request to retrieve the page. This is usually a GET /HTTP/1.1 request containing host: google.com and connection: close [other headers] where other headers comprise of single new lines of colon-separated series of key-value pairs in the format of the HTTP specification. HTTP/1.1 here is the signal showing that the connection closes as soon as the response completes.
Upon fulfilling these deliverables, the server then receives a single blank new line from the web browser, which indicates a completion of the request. Next, the server replies with a response code stating the status of the request and corresponding response code, in this case, 200 OK. The server then decides to close the connection or keep it open
for more requests depending on whether the client’s header requested this information. After the HTML portion is parsed, this process is repeated for every other resource that is referenced within the HTML, which includes CSS, JS, image, and media. In this case, instead of GET / HTTP1.1, the request becomes GET /$(URL)HTTP/1.1. In some cases, the HTML can reference a resource hosted on another domain, then the browser repeats the process involving DNS modification of the hostname.
HTTP Server Request Handle
On the server-side, the HTTPD server or HTTP Daemon (which can be Apache or Nginx for Linux and IIS for Windows) handles all requests or responses from the client. As soon as the HTTPD receives the request,
The server breaks down the incoming request into three parameters;
HTTP Request Method (GET), Domain (google.com) and requested path (index, since no path/page is specified after “/”). The servers then verify the availability of a remote virtual host configured to resolve to google.com and verifies that the domain can accept GET requests. If the server is configured to use a rewrite module (URL Rewrite in IIS or mod_rewrite in Apache), the existing rewrite rule in the module is utilized, else, the server continues to pull content housed in the index folder, treating the URL as https://www.google.com/. The server then parses the content of this file and interprets the content of the index file to a process that outputs a user-readable format of the content.
The HTML parser, using built-in rendering capability, audits the HTML markup into a parse tree, which is a tree of DOM element attribute nodes. Because HTML cannot be parsed chronologically, the browser is forced to use a custom HTML parsing technique whose algorithm is determined by the HTML5 specification.
After these deliverables are met, the web browser begins the process of fetching external resources linked to the page, in the form of the layout (CSS), embedded scripts (Ajax, JS), and media (visuals). This raw file is marked as “in-progress” and parsing scripts are marked as “deferred”, which are resources awaiting execution after parsing is complete. When the document load state is complete, a load event is fired. In a situation where the browser encounters an error while loading an invalid content, the error state is logged in browser console but not directly visible to the end-user.
CSS interpretation and Page rendering
Using standard CSS lexicon and syntax containing CSS rules with selectors and objects, resources are parsed into a stylesheet object. This is a stateless activity as CSS parsers operate top-down or bottom-up, depending on the parser generator in use. The browser renders the page by creating a ‘Frame Tree’ and calculating the CSS style values for each node. Furthermore, the browser then calculates the preferred width, actual width, height, coordinates, textures, wrapping, margins, borders, and padding of each node in the ‘Frame Tree’ bottom-up using the standard CSS set rules. Next, it creates layers that describe what parts of the page can be logically grouped for animation without being re-rasterized. The CPU can either rasterize the transversed render objects for each layer or drawn on the GPU directly, using D2D/SkiaGL for rendering on the backend.
During rendering, the graphical computing layers either make use of a general-purpose CPU or the graphical processor’s GPU for advanced rendering. In cases where a refresh or reload command is triggered, the pre-existing values are used, excluding complex web applications where interactive user fields are present. The browser then refines the output using logical composite commands, which are enforced using hardware acceleration drivers like Direct3D/OpenGL. The GPU command buffer flushes for asynchronous rendering, and the frame is forwarded. After rendering, the browser finally executes other applicable scripts, including Flash (deprecated), interactive hardware plugins, and the requested query displays.
Other websites involving a process excerpt from the CIA (confidentiality, Integrity, Availability) triad including extended input parsing, language translation, form submission, password authentication, proxy/firewall network, and web application integration require a more sophisticated approach.
Source: 5G exposed
It joined “Super Mario Kart,” “Microsoft Solitaire,” and “Colossal Cave Adventure” as part of the 2019 class. The four inductees span multiple decades, countries of origin, and gaming platforms, but all have significantly influenced the video game industry, popular culture, and society in general, The Strong said.
Published by Midway Games in 1992, “Mortal Kombat” is one of the most recognizable fighting games around. Its hyper violent content led to major controversy, several court cases, and the creation of the Entertainment Software Ratings Board. Since then, it’s become one of the biggest media properties of all time, spawning nearly a dozen sequels, a television series, comics, feature films, and more.
Twelve finalists were announced for the 2019 World Video Game Hall of Fame class. They reportedly received thousands of nominations from more than 100 countries. To be eligible, the games must have longevity, geographical reach, and influence. They also need to reach cultural icon status.
Here is the full list:
- “Candy Crush”
- “Colossal Cave Adventure”
- “Dance Dance Revolution”
- “Microsoft Windows Solitaire”
- “Mortal Kombat”
- “NBA 2K”
- “Sid Meier’s Civilization”
- “Super Mario Kart”
- “Super Smash Bros. Melee”
Previous inductees include “Donkey Kong,” “Doom,” “Final Fantasy VII,” “The Legend of Zelda,” and the “The Oregon Trail.”
Facebook and Instagram appear to be partially down for some users around the world today. While you can open both platforms, it looks like you can’t send or receive messages on either platform, and you can’t post new content either.
WhatsApp appeared to be fine for many people, but users in Paraguay, India, Bangladesh, Argentina, and more note that they are experiencing issues with sending messages. DownDetector indicates that those in Brazil were experiencing the most severe outages.
We tested multiple accounts at The Verge, and found that Messenger couldn’t load at all on desktop, although the mobile app was working. Instagram was worse: posts weren’t loading, Instagram Stories were down, and direct messages and the button to post new content were also not working. Facebook’s ad section was not functioning either, and it led to an internal error when you tried to buy an ad.
About an hour after users noted the outage, Facebook responded on Twitter. It also noted that “the issue is not related to a DDoS attack.”
Facebook on desktop as of 4PM ET.
According to DownDetector, it looks like the outages are mainly in New England; Texas; Seattle, Washington; parts of Latin America, including Peru; the UK; India; and the Philippines. Users have written in from Canada, Las Vegas, and Turkey to note outages there as well. We’ve reached out to Facebook and Instagram to learn more.
It now looks like Oculus is also down. One user in California wrote in to The Verge, “Nobody can log in to any multiplayer games purchased through the Oculus store. They also can’t access their Oculus Home environments.” Users also reported being unable to buy games from the Oculus store.
Victor E Garcia@otiteb
@oculus , i am having problems managing apps on the dashboard: users in release channel that were already in are not appearing now. Are your systems having some sort of issues today?
Hey there, we’re currently looking into an issue on our end, and hope to have it resolved soon. We really appreciate your patience. Thank you!
See Oculus Support’s other Tweets
Other users noted that attempting to use Facebook to sign into apps like Tinder or Spotify wasn’t working either. Attempting to do so would bring up an error saying this feature isn’t available right now. If you were already signed into Spotify, it appears that your login is still valid, but once you sign out, you’ll be unable to get back in.
Makena Kelly and Esther Cohen contributed to this report.
Update March 13th, 2:08PM ET: This article has been updated with further details and comment from Facebook. It now appears that WhatsApp is also down for some users.
Update March 13th, 4:05PM ET: Some users have reported Oculus is also down and that the feature for authenticating logins through Facebook is down.
Source: The verge
An emergency directive from the Department of Homeland Security provides “required actions” for U.S. government agencies to prevent widespread DNS hijacking attacks.
The Department of Homeland Security is ordering all federal agencies to urgently audit Domain Name System (DNS) security for their domains in the next 10 business days.
The department’s rare “emergency directive,” issued Tuesday, warned that multiple government domains have been targeted by DNS hijacking attacks, allowing attackers to redirect and intercept web and mail traffic.
“[The Cybersecurity and Infrastructure Security Agency] (CISA) is aware of multiple executive branch agency domains that were impacted by the tampering campaign and has notified the agencies that maintain them,” said the alert.
The warning comes on the heels of a Jan. 10 FireEye report which detailed a wave of DNS hijacking attacks targeting victims in North America, Europe, Middle East and North Africa.
DNS hijacking is a type of malicious attack in which an individual redirects queries to a domain name server via overriding a computer’s transmission control protocol/internet protocol (TCP/IP) settings – generally by modifying a server’s settings.
The DHS, for its part, said that the attacker begins by logging into the DNS provider’s administration panel using previously-compromised credentials.
The attacker then alters DNS records – including the address mail exchanger or name server records – and replaces the legitimate address of a service with their own address controls, thus redirecting traffic. Attackers can also alter and tamper with the traffic flows.
“This enables them to direct user traffic to their own infrastructure for manipulation or inspection before passing it on to the legitimate service, should they choose,” said the DHS in its advisory. “This creates a risk that persists beyond the period of traffic redirection.”
Since the attackers can set record values for the domain name systems, they can obtain valid encryption certificates for an organization’s domain names; this allows browsers to establish a connection without any certificate errors as the certificate can be trusted, FireEye researchers said. In the most recent campaigns, the attackers have used certificates from the Let’s Encrypt open certificate authority.
That valid certificate then enables the redirected traffic to be decrypted and exposes any user-submitted data.
The emergency directive issued by the DHS provides “required actions” that government agencies must fulfill in the next 10 business days.
“To address the significant and imminent risks to agency information and information systems presented by this activity, this emergency directive requires… near-term actions to mitigate risks from undiscovered tampering, enable agencies to prevent illegitimate DNS activity for their domains and detect unauthorized certificates,” said the report.
First, the DHS said all .gov domain admins must audit their DNS records over the next 10 days to verify if any traffic is being redirected.
The department also urged agencies to update their passwords for all accounts on systems that can make changes to agency DNS records, and to implement multi-factor authentication for accounts on DNS admin systems. Finally, agencies are being directed to monitor certificate transparency logs.
The warning comes as the U.S. government enters its 33rd day of a shutdown (as of Wednesday), a longstanding incident which has sparked concerns about its impact across the board when it comes to security.
Researchers assess “with moderate confidence” that the recent DNS hijacking activity is conducted by a group or groups in Iran, and that the activity aligns with Iranian government interests.
The attacks have been observed in clusters between January 2017 to January 2019, the researchers said in an analysis of the attacks.
Alister Shepherd, MEA director of Mandiant at FireEye, told Threatpost that the campaign is ongoing – but that there is no indication of how many credentials have been harvested thus far. However, researcher do state that the attackers had “a high degree of success” harvesting targets’ credentials.
This most recent DNS hijacking campaign “showcases the continuing evolution in tactics from Iran-based actors,” FireEye researchers stressed. “This is an overview of one set of TTPs that we recently observed affecting multiple entities.”
A British hacker whose cyberattacks took the nation of Liberia offline has been jailed for almost three years.
Daniel Kaye launched a series of attacks on Liberian cell phone operator Lonestar in October 2015, which became so powerful they knocked out the west African country’s internet the following year.
Kaye, 30, had been hired to carry out the attacks by a senior employee at rival operator Cellcom, Britain’s National Crime Agency said in a statement, although there is no suggestion that Cellcom was aware of the activity.
He pleaded guilty to creating and using a botnet, a series of computers connected in order to attack systems, and possessing criminal property last month. Kaye was sentenced on Friday at Blackfriars Crown Court in central London to two years and eight months in prison.
While living in Cyprus, Kaye used a botnet he had created to trigger repeated distributed denial of service (DDoS) requests on Lonestar, causing the company to spend around $600,000 in remedial action.
The additional impact of customers leaving the network caused the company to lose tens of millions of dollars in lost revenue, the NCA added.
Following his arrest in February 2017, Kaye was extradited to Germany, where he also admitted to attacks on Deutsche Telekom that affected around 1 million customers in November 2016.
“Daniel Kaye was operating as a highly skilled and capable hacker-for-hire,” Mike Hulett, Head of Operations at the NCA’s National Cyber Crime Unit, said.
“His activities inflicted substantial damage on numerous businesses in countries around the world, demonstrating the borderless nature of cyber crime,” he added. “The victims in this instance suffered losses of tens of millions of dollars and had to spend a large amount on mitigating action.”
Andrew Bustamante recently responded to a question on a subreddit thread on Reddit which asked what the biggest threat to the America national security in the future will be. He replied saying blockchain technology will be the biggest threat and whoever manages to figure out the technology first wins.
The subreddit thread where the question was raised is a thread dedicated to Bustamante’s ‘Everyday Espionage’, where he supposedly teaches people to benefit from international espionage techniques by implementing them in daily lives. Considering the fact that Bustamante is reportedly a former CIA intelligence officer, U.S. Air Force veteran and Fortune 10 corporate advisor, he does have the qualifications to do so.
On 22 December 2018, the question was raised in the subreddit thread, together with several possible answers such as “climate change”, “Iran”, “North Korea” or “Russia”. He responded saying blockchain technology is “super powerful stuff”, and the whoever is capable of figuring out “how to hack it, manipulate it or bring it down wins”. However, he did not elaborate further on how exactly blockchain could threaten the country’s security.
Since then, multiple Redditors have speculated possible theories regarding Bustamante’s remarks on the technology. One of them suggested that the ability to create false data within the blockchain would be disastrous when paired with the high difficulty of fixing data in a blockchain-based system. Another Redditor suggested that blockchain offers the potential of making certain transactions “untraceable” and “anonymous”, thus ending up being a possible threat to security.
Quantum computing was also mentioned in the thread. Bustamante replied to a Redditor’s question asking for clarification whether developments in blockchain or quantum computing would be a more concerning factor to the country’s security. He replied that the Redditor had a “fair point”, and this remark a lot sparked new discussions on the topic.
As quantum computing was said to hold the possibility of permanently changing the cryptographic protection that defines blockchain, it was speculated that quantum computing was what Bustamante was referring to by his “bring it down” remark.
Over the past few months, the U.S. Department of Justice (DoJ) had charged several people from Russia’s Main Intelligence Directorate (GRU) for crypto-related crimes. In July, the DoJ charged twelve individuals while another seven officers were charged in October, all from the GRU and involved in crypto-related hacking operations.