Blogs or Expert Columns
Azure HDInsight Spark
Azure HDInsight Spark Overview Introduction to Spark on HDInsight This article provides you with an introduction to Spark on HDInsight. Apache Spark is an open-source parallel processing framework that supports in-memory processing to boost the performance of big-data analytic applications. Spark cluster on HDInsight is compatible with Azure Storage (WASB) as well as Azure Data Lake Store. Hence, your existing data stored in Azure can easily be processed via a Spark cluster. When you create a Spark cluster on HDInsight, you create Azure compute resources with Spark installed and configured. It only takes about 10 minutes to create a Spark cluster [...]
Compression
Compression is a simple, effective way to save bandwidth and speed up your site. It’s the 21st century. Most of the traffic comes from modern browsers, and quite frankly, most of the users are fairly tech-savvy. No one wants to slow someone else down because somebody is chugging along on IE 4.0 on Windows 95. Google and Yahoo use gzip compression. A modern browser is needed to enjoy modern web content and modern web speed — so gzip encoding it is. Here’s how to set it up. Before we start we should explain what content encoding is. When you request a [...]
Amazon Rekognition
What Is Amazon Rekognition? Amazon Rekognition makes it easy to add image and video analysis to your applications. You just provide an image or video to the Rekognition API, and the service can identify objects, people, text, scenes, and activities. It can detect any inappropriate content as well. Amazon Rekognition also provides highly accurate facial analysis and facial recognition. You can detect, analyze, and compare faces for a wide variety of use cases, including user verification, cataloging, people counting, and public safety. Amazon Rekognition is based on the same proven, highly scalable, deep learning technology developed by Amazon’s computer [...]
Amazon S3( Simple Storage Service)
What Is Amazon S3? Amazon Simple Storage Service is storage for the Internet. It is designed to make web-scale computing easier for developers. Amazon S3 has a simple web services interface that you can use to store and retrieve any amount of data, at any time, from anywhere on the web. It gives any developer access to the same highly scalable, reliable, fast, inexpensive data storage infrastructure that Amazon uses to run its own global network of web sites. The service aims to maximize benefits of scale and to pass those benefits on to developers. This guide explains the [...]
Apache Sentry
What is Apache Sentry? Apache Sentry is a granular, role-based authorization module for Hadoop. Sentry provides the ability to control and enforce precise levels of privileges on data for authenticated users and applications on a Hadoop cluster. Sentry currently works out of the box with Apache Hive, Hive Metastore/HCatalog, Apache Solr, Impala and HDFS (limited to Hive table data). Sentry is designed to be a pluggable authorization engine for Hadoop components. It allows you to define authorization rules to validate a user or application’s access requests for Hadoop resources. Sentry is highly modular and can support authorization for a [...]
Amazon AWS
What Is Amazon AWS? Amazon AWS is the implementation of Amazon Web Services in the Beijing and Ningxia (China) Regions. Directly connected to domestic telecommunication networks, it's designed to provide the infrastructure and network services necessary to support AWS technology. Amazon AWS provides computing resources and services that you can use to quickly and cost-effectively build applications. For example, you can rent a virtual server on Amazon AWS that you can connect to, configure, secure, and run just as you would a physical server. But the virtual server runs on top of a global network managed by AWS, and [...]
Security
Web server security is the protection of information assets that can be accessed from a Web server. Web server security is important for any organization that has a physical or virtual Web server connected to the Internet. It requires a layered defense and is especially important for organizations with customer-facing websites. Separate servers should be used for internal and external-facing applications and servers for external-facing applications should be hosted on a DMZ or containerized service network to prevent an attacker from exploiting a vulnerability to gain access to sensitive internal information. Penetration tests should be run on a regular basis to identify potential attack vectors, which are often caused by out-of-date server modules, configuration or coding [...]
How to Use MySQL
Overview This Document explains some of the basic SQL statements. If this is the first time you have used a relational database management system, this tutorial gives you everything you need to know to work with MySQL such as querying data, updating data, managing databases, and creating tables. If you’re already familiar with other relational database management systems such as PostgreSQL, Oracle, or Microsoft SQL Server, etc., you can use this tutorial to refresh your knowledge and understand how SQL dialect of MySQL is different from other systems. Section 1. Getting started with MySQL This section helps you get [...]
SAS
SAS stands for Statistical Analysis Software. It was created in the year 1960 by the SAS Institute. From 1st January 1960, SAS was used for data management, business intelligence, Predictive Analysis, Descriptive and Prescriptive Analysis etc. Since then, many new statistical procedures and components were introduced in the software The core of the SAS System is base SAS software, which consists of SAS language a programming language that you use to manage your data.SAS procedures software tools for data analysis and reporting.macro facility a tool for extending and customizing SAS software programs and for reducing the text in your [...]
SQL
Overview SQL (Structured Query Language) is used to modify and access data or information from a storage area called database. This beginner online training SQL tutorial website teaches you the basics of SQL code and trains you how to write & program SQL queries. I will be sharing my database knowledge on SQL and help you learn programming SQL better. The concepts discussed in this SQL tutorial can be applied to most of the database systems. The SQL syntax used to explain the tutorial concepts is similar to the one used in Oracle database. What is SQL? What is SQL? SQL stands for “Structured Query [...]
Power BI
What is Power BI? Power BI is a suite of business analytics tools to analyze data and share insights. Power BI dashboards provide a 360-degree view of business users with their most important metrics in one place, updated in real time, and available on all of their devices. With one click, users can explore the data behind their dashboard using intuitive tools that make finding answers easy. Creating a dashboard is simple, thanks to hundreds of connections to popular business applications, complete with pre-built dashboards to help you get up and running quickly. And you can access your data and reports from anywhere [...]
Azure Machine Learning
Azure Machine Learning Azure Machine Learning is an integrated, end-to-end data science and advanced analytics solution. It enables data scientists to prepare data, develop experiments, and deploy models at cloud scale. The main components of Azure Machine Learning are: Azure Machine Learning Workbench Azure Machine Learning Experimentation Service Azure Machine Learning Model Management Service Microsoft Machine Learning Libraries for Apache Spark (MMLSpark Library) Visual Studio Code Tools for AI Together, these applications and services help significantly accelerate your data science project development and deployment. Open source compatible Azure Machine Learning fully supports open source technologies. You can use tens [...]
Azure Bot Service
Azure Bot Service: Azure Bot Service provides what you need to build, connect, test, deploy, monitor, and manage bots. Bot Service provides the core components for creating bots, including the Bot Builder SDK for developing bots and the Bot Framework for connecting bots to channels. Bot Service provides an integrated environment purpose-built for bot development. You can write a bot, connect, test, deploy, and manage it from your web browser with no separate editor or source control required. For simple bots, you may not need to write code at all. It is powered by the Bot Framework and it provides [...]
Azure Emotion API
Welcome to the Microsoft Emotion API, which allows you to build more personalized apps with Microsoft’s cutting-edge cloud-based emotion recognition algorithm. Emotion Recognition The Emotion API beta takes an image as an input, and returns the confidence across a set of emotions for each face in the image, as well as the bounding box for the face, from the Face API. The emotions detected are happiness, sadness, surprise, anger, fear, contempt, disgust or neutral. These emotions are communicated cross-culturally and universally via the same basic facial expressions, where are identified by Emotion API. Interpreting Results: In [...]
Azure Redis Cache
Azure Redis Cache: Azure Redis Cache is a distributed, managed cache that helps you build highly scalable and responsive applications by providing super-fast access to your data. The new Premium-tier is an Enterprise ready tier, which includes all the Standard-tier features and more, such as better performance, bigger workloads, disaster recovery, import/export, and enhanced security. Continue reading to learn more about the additional features of the Premium cache tier. Better performance compared to Standard or Basic Tier Better performance over Standard or Basic tier. Caches in the Premium tier are deployed on hardware which has faster processors and gives better [...]
Predictive Policing: The Future of Law Enforcement
The era of predictive analytics has arrived. And it has the potential to equip police departments and citizens around the world with the intelligence they need to predict crime both in real time and in the future. Thanks to everything from automatic license plate readers to gun sensors, the real-time data available to law enforcement agencies continue to skyrocket. And with Microsoft’s advanced analytics capabilities such as Microsoft Power BI, Microsoft Azure Stream Analytics, and Microsoft Azure Machine Learning (Azure ML), police departments now have the capability to predict when and where crimes will happen in the future. By building a crime analytics [...]
Azure SQL Database
Azure SQL Database SQL Database is a general-purpose relational database service in Microsoft Azure that supports structures such as relational data, JSON, spatial, and XML. It delivers dynamically scalable performance and provides options such as columnstore indexes for extreme analytic analysis and reporting and in-memory OLTP for extreme transactional processing. Microsoft handles all patching and updating of the SQL code base seamlessly and abstracts away all management of the underlying infrastructure. SQL Database shares its code base with the Microsoft SQL Server database engine. With Microsoft's cloud-first strategy, the newest capabilities of SQL Server are released first to SQL Database, [...]
Azure Data Lake Store
Azure Data Lake Store Azure Data Lake Store is an enterprise-wide hyper-scale repository for big data analytics workloads. Azure Data Lake enables you to capture data of any size, type, and ingestion speed in one single place for operational and exploratory analytics. Azure Data Lake Store can be accessed from Hadoop (available with HDInsight cluster) using the WebHDFS-compatible REST APIs. It is specifically designed to enable analytics on the stored data and is tuned for performance for data analytics scenarios. Out of the box, it includes all the enterprise-grade capabilities—security, manageability, scalability, reliability, and availability—essential for real-world enterprise use [...]
Azure SQL Data Warehouse
Azure SQL Data Warehouse SQL Data Warehouse is a cloud-based Enterprise Data Warehouse (EDW) that leverages Massively Parallel Processing (MPP) to quickly run complex queries across petabytes of data. Use SQL Data Warehouse as a key component of a big data solution. Import big data into SQL Data Warehouse with simple PolyBase T-SQL queries, and then use the power of MPP to run high-performance analytics. As you integrate and analyze, the data warehouse will become the single version of truth your business can count on for insights. Key component of big data solution SQL Data Warehouse is a [...]
Azure Cosmos DB
Azure Cosmos DB Azure Cosmos DB is Microsoft's globally distributed, multi-model database. With the click of a button, Azure Cosmos DB enables you to elastically and independently scale throughput and storage across any number of Azure's geographic regions. It offers throughput, latency, availability, and consistency guarantees with comprehensive service level agreements (SLAs), something no other database service can offer. Key capabilities As a globally distributed, multi-model database service, Azure Cosmos DB makes it easy to build scalable, highly responsive applications at the global scale: Turnkey global distribution You can distribute your data to any number of Azure regions, with the click of a [...]
Azure Event Hub
Azure Event Hub Azure Event Hubs is a highly scalable data streaming platform and event ingestion service, capable of receiving and processing millions of events per second. Event Hubs can process and store events, data, or telemetry produced by distributed software and devices. Data sent to an event hub can be transformed and stored using any real-time analytics provider or batching/storage adapters. With the ability to provide publish-subscribe capabilities with low latency and at massive scale, Event Hubs serves as the "on-ramp" for Big Data. Why use Event Hubs? Event Hubs event and telemetry handling capabilities make it especially [...]
Azure Data Factory
Azure Data Factory In the world of big data, raw, unorganized data is often stored in relational, non-relational, and other storage systems. However, on its own, raw data doesn't have the proper context or meaning to provide meaningful insights to analysts, data scientists, or business decision makers. Big data requires service that can orchestrate and operationalize processes to refine these enormous stores of raw data into actionable business insights. Azure Data Factory is a managed cloud service that's built for these complex hybrid extract-transform-load (ETL), extract-load-transform (ELT), and data integration projects. For example, imagine a gaming company that collects [...]
Azure Stream Analytics
Azure Stream Analytics Azure Stream Analytics is a managed event-processing engine set up real-time analytic computations on streaming data. The data can come from devices, sensors, websites, social media feeds, applications, infrastructure systems, and more. Use Stream Analytics to examine high volumes of data streaming from devices or processes, extract information from that data stream, identify patterns, trends, and relationships. Use those patterns to trigger other processes or actions, like alerts, automation workflows, feed information to a reporting tool, or store it for later investigation. Some examples: Stock-trading analysis and alerts. Fraud detection, data, and identify protections. Embedded sensor [...]
Azure HDInsight
Azure HDInsight Azure HDInsight is a fully managed, full-spectrum, open-source analytics service for enterprises. HDInsight is a cloud service that makes it easy, fast, and cost-effective to process massive amounts of data. HDInsight also supports a broad range of scenarios, like extract, transform, and load (ETL); data warehousing; machine learning; and IoT. What is HDInsight and the Hadoop technology stack? Azure HDInsight is a cloud distribution of the Hadoop components from the Hortonworks Data Platform (HDP). Azure HDInsight makes it easy, fast, and cost-effective to process massive amounts of data. You can use the most popular open-source frameworks such as [...]
