Class: Aws::QConnect::Types::WebCrawlerConfiguration

Inherits:

Struct

• Object
• Struct
• Aws::QConnect::Types::WebCrawlerConfiguration

Defined in:

gems/aws-sdk-qconnect/lib/aws-sdk-qconnect/types.rb

Overview

The configuration details for the web data source.

See Also:

• AWS API Documentation

Constant Summary

SENSITIVE =

[:inclusion_filters, :exclusion_filters]

Instance Attribute Summary

• #crawler_limits ⇒ Types::WebCrawlerLimits

  The configuration of crawl limits for the web URLs.

• #exclusion_filters ⇒ Array<String>

  A list of one or more exclusion regular expression patterns to exclude certain URLs.

• #inclusion_filters ⇒ Array<String>

  A list of one or more inclusion regular expression patterns to include certain URLs.

• #scope ⇒ String

  The scope of what is crawled for your URLs.

• #url_configuration ⇒ Types::UrlConfiguration

  The configuration of the URL/URLs for the web content that you want to crawl.

Instance Attribute Details

#crawler_limits ⇒ Types::WebCrawlerLimits

The configuration of crawl limits for the web URLs.

Returns:

• (Types::WebCrawlerLimits)

# File 'gems/aws-sdk-qconnect/lib/aws-sdk-qconnect/types.rb', line 12459

class WebCrawlerConfiguration < Struct.new(
  :url_configuration,
  :crawler_limits,
  :inclusion_filters,
  :exclusion_filters,
  :scope)
  SENSITIVE = [:inclusion_filters, :exclusion_filters]
  include Aws::Structure
end

#exclusion_filters ⇒ Array<String>

A list of one or more exclusion regular expression patterns to exclude certain URLs. If you specify an inclusion and exclusion filter/pattern and both match a URL, the exclusion filter takes precedence and the web content of the URL isn’t crawled.

Returns:

• (Array<String>)

# File 'gems/aws-sdk-qconnect/lib/aws-sdk-qconnect/types.rb', line 12459

class WebCrawlerConfiguration < Struct.new(
  :url_configuration,
  :crawler_limits,
  :inclusion_filters,
  :exclusion_filters,
  :scope)
  SENSITIVE = [:inclusion_filters, :exclusion_filters]
  include Aws::Structure
end

#inclusion_filters ⇒ Array<String>

A list of one or more inclusion regular expression patterns to include certain URLs. If you specify an inclusion and exclusion filter/pattern and both match a URL, the exclusion filter takes precedence and the web content of the URL isn’t crawled.

Returns:

• (Array<String>)

# File 'gems/aws-sdk-qconnect/lib/aws-sdk-qconnect/types.rb', line 12459

class WebCrawlerConfiguration < Struct.new(
  :url_configuration,
  :crawler_limits,
  :inclusion_filters,
  :exclusion_filters,
  :scope)
  SENSITIVE = [:inclusion_filters, :exclusion_filters]
  include Aws::Structure
end

#scope ⇒ String

The scope of what is crawled for your URLs. You can choose to crawl only web pages that belong to the same host or primary domain. For example, only web pages that contain the seed URL https://docs.aws.amazon.com/bedrock/latest/userguide/ and no other domains. You can choose to include sub domains in addition to the host or primary domain. For example, web pages that contain aws.amazon.com can also include sub domain docs.aws.amazon.com.

Returns:

• (String)

# File 'gems/aws-sdk-qconnect/lib/aws-sdk-qconnect/types.rb', line 12459

class WebCrawlerConfiguration < Struct.new(
  :url_configuration,
  :crawler_limits,
  :inclusion_filters,
  :exclusion_filters,
  :scope)
  SENSITIVE = [:inclusion_filters, :exclusion_filters]
  include Aws::Structure
end

#url_configuration ⇒ Types::UrlConfiguration

The configuration of the URL/URLs for the web content that you want to crawl. You should be authorized to crawl the URLs.

Returns:

• (Types::UrlConfiguration)

# File 'gems/aws-sdk-qconnect/lib/aws-sdk-qconnect/types.rb', line 12459

class WebCrawlerConfiguration < Struct.new(
  :url_configuration,
  :crawler_limits,
  :inclusion_filters,
  :exclusion_filters,
  :scope)
  SENSITIVE = [:inclusion_filters, :exclusion_filters]
  include Aws::Structure
end